Nov 03 16:09:37.036024 ip-10-0-142-22 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 16:09:37.490878 ip-10-0-142-22 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 16:09:37.490878 ip-10-0-142-22 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 16:09:37.490878 ip-10-0-142-22 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 16:09:37.490878 ip-10-0-142-22 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 16:09:37.495010 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.494931    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:09:37.495010 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495010    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:09:37.495010 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495014    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495018    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495021    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495024    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495027    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495035    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495038    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495041    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495044    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495047    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495050    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495052    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495055    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495058    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495060    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495063    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495065    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495068    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495071    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495073    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:09:37.495099 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495076    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495079    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495081    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495084    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495086    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495089    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495091    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495094    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495097    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495099    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495102    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495105    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495108    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495110    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495113    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495116    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495119    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495122    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495125    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495128    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:09:37.495570 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495130    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495133    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495135    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495138    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495140    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495143    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495145    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495147    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495150    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495152    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495155    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495157    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495160    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495165    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495170    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495173    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495177    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495180    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495183    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495185    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:09:37.496096 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495188    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495191    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495195    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495198    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495202    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495205    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495208    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495210    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495214    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495217    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495220    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495223    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495226    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495228    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495231    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495233    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495236    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495238    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:09:37.496567 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495241    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495243    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495246    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495248    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495251    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495254    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495257    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495268    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495272    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495275    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495278    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495281    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495283    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495286    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495289    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495291    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495294    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495296    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495299    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495301    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:09:37.497004 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495304    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495306    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495309    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495313    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495315    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.495319    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497045    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497054    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497058    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497061    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497065    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497068    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497071    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497075    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497078    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497080    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497083    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497086    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497089    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497091    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:09:37.497467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497094    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497097    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497099    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497102    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497104    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497107    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497109    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497112    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497114    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497117    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497119    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497122    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497124    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497126    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497130    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497133    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497135    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497138    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497141    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:09:37.497964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497144    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497147    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497150    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497152    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497155    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497158    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497160    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497163    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497165    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497168    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497172    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497176    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497179    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497183    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497187    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497190    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497193    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497196    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497199    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:09:37.498420 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497201    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497204    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497206    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497209    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497211    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497214    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497216    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497219    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497224    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497226    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497229    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497231    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497234    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497237    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497240    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497242    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497245    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497248    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497250    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497253    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:09:37.498886 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497256    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497258    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497261    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497263    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497266    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497269    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497271    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497274    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497277    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497279    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497282    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497284    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497287    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497289    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497291    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497294    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497297    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497299    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497302    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497304    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:09:37.499380 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497308    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497311    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497314    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497317    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497320    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497323    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497326    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497329    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497331    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497334    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497337    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497339    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497343    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497345    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497856    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497862    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497865    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497868    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497871    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497873    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:09:37.499903 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497876    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497879    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497881    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497884    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497887    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497889    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497892    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497894    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497897    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497899    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497902    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497904    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497908    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497911    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497913    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497916    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497919    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497922    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497925    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497928    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:09:37.500417 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497931    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497933    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497937    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497940    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497943    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497945    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497948    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497951    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497953    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497956    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497959    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497962    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497964    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497967    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497970    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497973    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497975    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497978    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:09:37.501104 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497981    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497983    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497986    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497988    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497991    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497993    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497996    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.497999    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498002    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498004    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498007    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498009    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498012    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498021    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498024    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498027    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498030    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498032    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498035    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498038    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:09:37.501642 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498040    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498043    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498045    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498048    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498051    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498053    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498056    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498059    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498061    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498064    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498066    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498069    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498071    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498074    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498076    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498079    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498081    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498084    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498088    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498090    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:09:37.502216 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498093    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498096    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498099    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498101    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498104    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498108    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498111    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498113    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498116    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498118    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498122    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498126    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498129    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498132    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498134    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498137    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498140    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498142    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498145    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:09:37.502767 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498147    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498150    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498154    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498241    2582 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498248    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498253    2582 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498260    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498266    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498271    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498276    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498280    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498284    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498287    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498290    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498294    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498297    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498300    2582 flags.go:64] FLAG: --cgroup-root=""
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498302    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498305    2582 flags.go:64] FLAG: --client-ca-file=""
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498308    2582 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498312    2582 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498316    2582 flags.go:64] FLAG: --cluster-domain=""
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498319    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498322    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 16:09:37.503310 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498325    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498328    2582 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498332    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498335    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498339    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498342    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498345    2582 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498348    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498351    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498354    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498357    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498361    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498364    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498367    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498370    2582 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498373    2582 flags.go:64] FLAG: --enable-server="true"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498376    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498379    2582 flags.go:64] FLAG: --event-burst="100"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498382    2582 flags.go:64] FLAG: --event-qps="50"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498385    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498388    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498391    2582 flags.go:64] FLAG: --eviction-hard=""
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498395    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498398    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498401    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 16:09:37.503954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498404    2582 flags.go:64] FLAG: --eviction-soft=""
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498407    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498410    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498412    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498416    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498419    2582 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498422    2582 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498425    2582 flags.go:64] FLAG: --feature-gates=""
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498429    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498432    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498435    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498438    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498441    2582 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498444    2582 flags.go:64] FLAG: --help="false"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498448    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498451    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498454    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498457    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498460    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498463    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498466    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498469    2582 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498472    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 16:09:37.504646 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498474    2582 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498477    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498480    2582 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498483    2582 flags.go:64] FLAG: --kube-reserved=""
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498486    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498489    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498492    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498494    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498497    2582 flags.go:64] FLAG: --lock-file=""
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498500    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498503    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498506    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498511    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498514    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498517    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498520    2582 flags.go:64] FLAG: --logging-format="text"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498523    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498527    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498529    2582 flags.go:64] FLAG: --manifest-url=""
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498532    2582 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498537    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498539    2582 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498543    2582 flags.go:64] FLAG: --max-pods="110"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498546    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498550    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 16:09:37.505433 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498553    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498556    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498559    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498561    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498564    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498571    2582 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498574    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498577    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498581    2582 flags.go:64] FLAG: --pod-cidr=""
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498583    2582 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498588    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498591    2582 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498594    2582 flags.go:64] FLAG: --port="10250"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498597    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498600    2582 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0442dc01194780f5f"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498603    2582 flags.go:64] FLAG: --qos-reserved=""
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498606    2582 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498609    2582 flags.go:64] FLAG: --register-node="true"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498612    2582 flags.go:64] FLAG: --register-with-taints=""
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498616    2582 flags.go:64] FLAG: --registry-burst="10"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498619    2582 flags.go:64] FLAG: --registry-qps="5"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498621    2582 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498625    2582 flags.go:64] FLAG: --reserved-memory=""
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498629    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498632    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498635    2582 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 16:09:37.506232 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498638    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498641    2582 flags.go:64] FLAG: --runonce="false"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498644    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498647    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498652    2582 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498655    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498658    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498661    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498664    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498667    2582 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498670    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498673    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498676    2582 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498678    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498681    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498684    2582 flags.go:64] FLAG: --system-cgroups=""
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498688    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498693    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498696    2582 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498698    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498702    2582 flags.go:64] FLAG: --tls-min-version=""
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498705    2582 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498707    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498710    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498713    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 16:09:37.506873 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498716    2582 flags.go:64] FLAG: --v="2"
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498719    2582 flags.go:64] FLAG: --version="false"
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498741    2582 flags.go:64] FLAG: --vmodule=""
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498745    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.498750    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498890    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498894    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498897    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498900    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498903    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498906    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498910    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498912    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498915    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498918    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498920    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498923    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498926    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498929    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498931    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:09:37.507600 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498934    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498936    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498939    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498943    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498945    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498948    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498950    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498953    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498955    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498958    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498960    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498964    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498968    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498971    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498973    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498976    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498978    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498984    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498987    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:09:37.508089 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498990    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498992    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498995    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.498997    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499001    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499004    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499006    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499009    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499012    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499014    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499017    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499019    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499022    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499025    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499027    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499030    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499034    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499036    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499039    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:09:37.508551 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499041    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499045    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499049    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499053    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499056    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499058    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499061    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499063    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499066    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499069    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499071    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499074    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499077    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499079    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499082    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499084    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499087    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499091    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499094    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:09:37.509019 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499096    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499099    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499102    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499104    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499107    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499109    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499112    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499114    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499117    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499120    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499123    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499126    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499128    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499132    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499134    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499136    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499139    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499141    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499144    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:09:37.509467 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499146    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499149    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499151    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499154    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499156    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499159    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499161    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499164    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499167    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499169    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499172    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499175    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499178    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499181    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499184    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.499209    2582 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 16:09:37.509946 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499246    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499251    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499254    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499257    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499267    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499270    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499273    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499277    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499279    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499282    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499285    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499287    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499290    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499293    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499295    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499298    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499300    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499303    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499305    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499309    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:09:37.510336 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499312    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499314    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499317    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499319    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499323    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499325    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499328    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499331    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499335    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499337    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499340    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499342    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499345    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499348    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499350    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499353    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499356    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499358    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499361    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499364    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:09:37.510821 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499367    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499370    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499372    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499374    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499377    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499380    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499382    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499385    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499389    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499392    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499396    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499399    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499401    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499404    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499406    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499409    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499412    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499415    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499417    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:09:37.511293 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499420    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499423    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499426    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499429    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499431    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499435    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499438    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499440    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499443    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499446    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499448    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499451    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499456    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499458    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499460    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499463    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499465    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499469    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499473    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499476    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:09:37.511758 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499479    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499481    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499484    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499486    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499489    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499492    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499494    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499497    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499499    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499502    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499504    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499507    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499511    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499515    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499517    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499520    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499522    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499525    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499527    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499530    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:09:37.512231 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499532    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499535    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499537    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499540    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499544    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499546    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.499549    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.509132    2582 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.509148    2582 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.509879    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.509887    2582 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509947    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509953    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509956    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509959    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509962    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509965    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509968    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509971    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:09:37.512760 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509974    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509976    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509979    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509982    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509984    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509987    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509991    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509993    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509996    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.509999    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510001    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510004    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510006    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510009    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510011    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510014    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510017    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510020    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510023    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510025    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:09:37.513212 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510028    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510031    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510034    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510037    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510040    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510042    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510045    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510047    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510051    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510056    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510059    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510063    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510066    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510068    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510071    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510074    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510076    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510079    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510083    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:09:37.513716 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510085    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510088    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510091    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510093    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510096    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510099    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510102    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510104    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510107    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510110    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510112    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510115    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510118    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510120    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510124    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510132    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510135    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510138    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510140    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510143    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:09:37.514182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510146    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510148    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510151    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510153    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510156    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510159    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510161    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510164    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510166    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510169    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510171    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510173    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510177    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510179    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510182    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510184    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510187    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510189    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510193    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:09:37.514915 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510196    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510199    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510202    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510204    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510207    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510210    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510212    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510215    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510218    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510220    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510223    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510226    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510228    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510230    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510233    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510235    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510238    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510240    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510242    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:09:37.515387 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510245    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.510273    2582 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510411    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510416    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510420    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510423    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510426    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510430    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510432    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510435    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510437    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510440    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510443    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510445    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510448    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510450    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:09:37.515836 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510452    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510455    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510458    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510461    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510464    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510467    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510470    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510472    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510475    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510477    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510480    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510482    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510485    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510488    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510490    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510494    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510497    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510500    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510503    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510506    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:09:37.516232 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510508    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510511    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510514    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510516    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510519    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510522    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510524    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510527    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510530    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510532    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510535    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510537    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510539    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510542    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510544    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510547    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510550    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510553    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510556    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:09:37.516692 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510560    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510564    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510567    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510570    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510572    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510575    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510577    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510580    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510582    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510584    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510587    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510589    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510592    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510594    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510597    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510600    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510602    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510605    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510607    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510610    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:09:37.517182 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510612    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510615    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510617    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510620    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510622    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510625    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510627    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510630    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510633    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510636    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510639    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510641    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510644    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510646    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510649    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510652    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510654    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510657    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510659    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:09:37.517657 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510662    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510665    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510667    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510670    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510672    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510675    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510677    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510680    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510682    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510685    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510687    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510689    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510692    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510695    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.510717    2582 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510789    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:09:37.518135 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510793    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510796    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510799    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510802    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510804    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510807    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510812    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510815    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510818    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510820    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510823    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510826    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510829    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510833    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510837    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510840    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510843    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510846    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510848    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:09:37.518510 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510851    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510853    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510856    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510858    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510861    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510863    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510866    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510868    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510871    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510873    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510876    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510879    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510881    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510884    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510887    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510889    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510892    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510895    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510898    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510902    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:09:37.519426 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510905    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510907    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510910    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510912    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510915    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510917    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510920    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510923    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510925    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510928    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510931    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510933    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510936    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510938    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510941    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510943    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510946    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510948    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510951    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:09:37.520060 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510954    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510957    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510959    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510962    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510966    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510969    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510971    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510974    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510976    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510979    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510981    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510984    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510987    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510990    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510993    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510995    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.510998    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511000    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511003    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:09:37.520511 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511005    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511008    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511010    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511013    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511015    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511018    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511021    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511024    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511027    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511029    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511031    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511034    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511037    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511039    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511042    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511044    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511047    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511050    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511054    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511058    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:09:37.521309 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511060    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511063    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511066    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511069    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511073    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511077    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511080    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:37.511082    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.511142    2582 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.513091    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.514378    2582 server.go:1014] "Starting client certificate rotation"
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.514490    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.514548    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.516922    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 16:09:37.522020 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.518063    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 16:09:37.531057 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.531041    2582 log.go:25] "Validated CRI v1 runtime API"
Nov 03 16:09:37.535772 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.535720    2582 log.go:25] "Validated CRI v1 image API"
Nov 03 16:09:37.536546 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.536518    2582 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 16:09:37.539179 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.539160    2582 fs.go:135] Filesystem UUIDs: map[4db0fe6b-a6a2-4b34-975a-22cd5ccc7aca:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bfe799c6-b6be-4cc8-83db-43b8a44f5da5:/dev/nvme0n1p3]
Nov 03 16:09:37.539229 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.539179    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 03 16:09:37.545052 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.544946    2582 manager.go:217] Machine: {Timestamp:2026-11-03 16:09:37.542697106 +0000 UTC m=+0.375337769 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101431 MemoryCapacity:32812023808 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec289dff6cc424a9cf7ba9283ee178b4 SystemUUID:ec289dff-6cc4-24a9-cf7b-a9283ee178b4 BootID:ce1434b7-7734-4e5d-bfc9-d4797dba3384 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406011904 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:e6:94:04:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:e6:94:04:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:dd:7c:db:1b:f7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812023808 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 16:09:37.545052 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.545051    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 16:09:37.545172 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.545160    2582 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 16:09:37.548587 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.548562    2582 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 16:09:37.548719 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.548591    2582 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-22.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 16:09:37.549328 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.549318    2582 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 16:09:37.549363 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.549331    2582 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 16:09:37.549363 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.549357    2582 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 16:09:37.549416 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.549370    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 16:09:37.550255 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.550246    2582 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 16:09:37.553300 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.553287    2582 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 16:09:37.553419 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.553410    2582 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 16:09:37.554172 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.554163    2582 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 16:09:37.554207 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.554180    2582 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 16:09:37.554207 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.554190    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 16:09:37.554207 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.554199    2582 kubelet.go:405] "Adding apiserver pod source"
Nov 03 16:09:37.554207 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.554207    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 16:09:37.554579 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.554566    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 16:09:37.556588 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.556569    2582 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 16:09:37.564018 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.564005    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 16:09:37.564922 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.564903    2582 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 16:09:37.564961 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.564933    2582 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 16:09:37.565091 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.565080    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 16:09:37.565139 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.565096    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 16:09:37.565139 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.565103    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 16:09:37.565139 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.565109    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 16:09:37.565139 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.565115    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 16:09:37.565139 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.565121    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 16:09:37.566426 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.566289    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 16:09:37.566426 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.566297    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 16:09:37.566426 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.566304    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 16:09:37.566426 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.566310    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 16:09:37.566426 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.566324    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 16:09:37.566426 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.566333    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 16:09:37.568855 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.568845    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 16:09:37.568889 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.568857    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 16:09:37.572593 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.572579    2582 server.go:1263] "Started kubelet"
Nov 03 16:09:37.572671 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.572632    2582 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 16:09:37.572794 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.572702    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 16:09:37.572849 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.572820    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 16:09:37.573429 ip-10-0-142-22 systemd[1]: Started Kubernetes Kubelet.
Nov 03 16:09:37.574254 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.574009    2582 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 16:09:37.578404 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.578386    2582 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 16:09:37.579010 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.578956    2582 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 03 16:09:37.582266 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.582245    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 16:09:37.583027 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.583007    2582 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 16:09:37.583146 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.583127    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:37.583691 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.583673    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 16:09:37.583780 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.583676    2582 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 16:09:37.583780 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.583744    2582 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 16:09:37.583860 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.583845    2582 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 16:09:37.583860 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.583853    2582 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 16:09:37.584177 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.584155    2582 factory.go:153] Registering CRI-O factory
Nov 03 16:09:37.584177 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.584171    2582 factory.go:223] Registration of the crio container factory successfully
Nov 03 16:09:37.584247 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.584210    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 16:09:37.584247 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.584217    2582 factory.go:55] Registering systemd factory
Nov 03 16:09:37.584247 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.584222    2582 factory.go:223] Registration of the systemd container factory successfully
Nov 03 16:09:37.584247 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.584236    2582 factory.go:103] Registering Raw factory
Nov 03 16:09:37.584247 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.584244    2582 manager.go:1199] Started watching for new ooms in manager
Nov 03 16:09:37.585258 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.585139    2582 manager.go:319] Starting recovery of all containers
Nov 03 16:09:37.596683 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.596667    2582 manager.go:324] Recovery completed
Nov 03 16:09:37.600521 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.600508    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:09:37.602324 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.602305    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:09:37.602387 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.602338    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:09:37.602387 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.602348    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:09:37.602826 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.602813    2582 cpu_manager.go:226] "Starting" policy="none"
Nov 03 16:09:37.602826 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.602825    2582 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 16:09:37.602910 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.602840    2582 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 16:09:37.602910 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.602872    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:09:37.603596 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.603575    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:09:37.603657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.603604    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:09:37.603657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.603618    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:09:37.604092 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.604081    2582 policy_none.go:50] "Start"
Nov 03 16:09:37.604148 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.604097    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 16:09:37.604148 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.604109    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 16:09:37.605301 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.605287    2582 policy_none.go:44] "Start"
Nov 03 16:09:37.653339 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.653294    2582 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 16:09:37.658379 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.658364    2582 manager.go:341] "Starting Device Plugin manager"
Nov 03 16:09:37.659249 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.659229    2582 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 16:09:37.659249 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.659250    2582 server.go:87] "Starting device plugin registration server"
Nov 03 16:09:37.659488 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.659479    2582 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 16:09:37.659526 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.659490    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 16:09:37.659767 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.659601    2582 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 16:09:37.659767 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.659691    2582 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 16:09:37.659767 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.659700    2582 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 16:09:37.660311 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.660259    2582 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 16:09:37.660344 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.660314    2582 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:37.703665 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.703471    2582 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-22.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 03 16:09:37.704803 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.703495    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-22.us-west-2.compute.internal.18e494d123f3b5f5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-22.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-22.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 16:09:37.572550133 +0000 UTC m=+0.405190796,LastTimestamp:2026-11-03 16:09:37.572550133 +0000 UTC m=+0.405190796,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-22.us-west-2.compute.internal,}"
Nov 03 16:09:37.709894 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.709870    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 16:09:37.711200 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.711183    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 16:09:37.711200 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.711203    2582 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 16:09:37.711342 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.711220    2582 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 16:09:37.711342 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.711249    2582 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 03 16:09:37.759938 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.759878    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:09:37.760754 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.760713    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:09:37.760846 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.760778    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:09:37.760846 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.760796    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:09:37.760846 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.760827    2582 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.768139 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.768117    2582 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 03 16:09:37.768315 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.768285    2582 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-22.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 03 16:09:37.768414 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.768315    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-22.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 16:09:37.768474 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.768345    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-22.us-west-2.compute.internal.18e494d125ba0488  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-22.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-22.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-22.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 16:09:37.602323592 +0000 UTC m=+0.434964254,LastTimestamp:2026-11-03 16:09:37.602323592 +0000 UTC m=+0.434964254,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-22.us-west-2.compute.internal,}"
Nov 03 16:09:37.782686 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.782670    2582 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 16:09:37.782785 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.782747    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lmd5l"
Nov 03 16:09:37.812313 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.812292    2582 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"]
Nov 03 16:09:37.812401 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.812345    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:09:37.813581 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.813564    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:09:37.813670 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.813599    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:09:37.813670 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.813613    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:09:37.815064 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.815049    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:09:37.815206 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.815192    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.815255 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.815222    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:09:37.816573 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.816556    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:09:37.816668 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.816573    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:09:37.816668 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.816588    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:09:37.816668 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.816596    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:09:37.816668 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.816602    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:09:37.816668 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.816607    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:09:37.817853 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.817837    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.817918 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.817865    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:09:37.818840 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.818820    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:09:37.818915 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.818850    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:09:37.818915 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.818860    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:09:37.844572 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.844547    2582 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found" node="ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.847716 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.847699    2582 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found" node="ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.849457 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.849442    2582 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 16:09:37.849563 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.849546    2582 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 16:09:37.856842 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.856827    2582 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.856914 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.856849    2582 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-22.us-west-2.compute.internal\": node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:37.885133 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.885113    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.885252 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.885139    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/608951668150694fd6003da288232820-config\") pod \"kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"608951668150694fd6003da288232820\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.885252 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.885155    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.974650 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:37.974619    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:37.985901 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.985877    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/608951668150694fd6003da288232820-config\") pod \"kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"608951668150694fd6003da288232820\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.985975 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.985914    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/608951668150694fd6003da288232820-config\") pod \"kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"608951668150694fd6003da288232820\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.985975 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.985917    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.985975 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.985945    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.986070 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.985980    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:37.986070 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:37.985985    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:38.075053 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.074986    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:38.148768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.148742    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:38.151553 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.151537    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:38.175350 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.175326    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:38.276032 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.275999    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:38.377067 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.376984    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:38.477336 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.477307    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:38.513819 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.513796    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 03 16:09:38.514265 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.513989    2582 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 16:09:38.514265 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.514001    2582 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 16:09:38.514265 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.513982    2582 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a7ccfbe3a02a843d98ab5cd19bd85dc7-f84d5194377c4192.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-142-22.us-west-2.compute.internal.18e494d125ba4e73\": read tcp 10.0.142.22:38522->13.184.209.27:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-142-22.us-west-2.compute.internal.18e494d125ba4e73  default   5080 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-22.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-22.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-22.us-west-2.compute.internal,},FirstTimestamp:2026-11-03 16:09:37 +0000 UTC,LastTimestamp:2026-11-03 16:09:37.813604913 +0000 UTC m=+0.646245577,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-22.us-west-2.compute.internal,}"
Nov 03 16:09:38.578275 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.578235    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:38.679002 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.678932    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:38.779990 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.779965    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:38.787281 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.787244    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 16:04:38 +0000 UTC" deadline="2028-04-06 11:49:26.069427815 +0000 UTC"
Nov 03 16:09:38.787281 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.787279    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12475h39m47.282152667s"
Nov 03 16:09:38.810535 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.810519    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:09:38.811418 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.811404    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:09:38.811469 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.811432    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:09:38.811469 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:38.811448    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:09:38.880316 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.880280    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:38.980520 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:38.980452    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:09:39.034672 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.034646    2582 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 16:09:39.083647 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.083625    2582 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:39.191437 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.191414    2582 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 16:09:39.191579 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.191538    2582 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 16:09:39.192559 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.192543    2582 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:09:39.244575 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:39.244531    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod608951668150694fd6003da288232820.slice/crio-987d64b915ab5b9587adb18f4ec8af3d3d9179036bd21512ceebd2aa91a707fd WatchSource:0}: Error finding container 987d64b915ab5b9587adb18f4ec8af3d3d9179036bd21512ceebd2aa91a707fd: Status 404 returned error can't find the container with id 987d64b915ab5b9587adb18f4ec8af3d3d9179036bd21512ceebd2aa91a707fd
Nov 03 16:09:39.252120 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.250603    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 16:09:39.258456 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.258438    2582 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 16:09:39.499413 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.499329    2582 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 16:09:39.557623 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.557574    2582 apiserver.go:52] "Watching apiserver"
Nov 03 16:09:39.584326 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.584272    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 03 16:09:39.676354 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.676323    2582 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 16:09:39.676664 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.676640    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-tbpr9","kube-system/konnectivity-agent-pkn6d","kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-8k9vh","openshift-dns/node-resolver-j8mcw","openshift-multus/network-metrics-daemon-k5rqd","openshift-network-diagnostics/network-check-target-lqvng","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal","openshift-multus/multus-qgfrq","openshift-network-operator/iptables-alerter-864tf","openshift-ovn-kubernetes/ovnkube-node-qns9l","openshift-image-registry/node-ca-gfrfv","openshift-multus/multus-additional-cni-plugins-g2fnp"]
Nov 03 16:09:39.679136 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.679120    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:39.679217 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.679199    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:39.682071 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.682055    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:39.684461 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.684441    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.687632 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.687608    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.690157 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.690141    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:39.692582 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.692563    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:39.692665 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.692639    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:39.695144 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695123    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.695244 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695153    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.695244 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695171    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl446\" (UniqueName: \"kubernetes.io/projected/dadfa10c-198c-4f93-869f-377048834934-kube-api-access-pl446\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.695244 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695191    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d80b9f00-54a1-4809-9856-efb0ee40bd24-host\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:39.695412 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695250    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjvvq\" (UniqueName: \"kubernetes.io/projected/d80b9f00-54a1-4809-9856-efb0ee40bd24-kube-api-access-sjvvq\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:39.695412 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695315    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-cnibin\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.695412 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.695606 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695437    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:39.695606 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695494    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d80b9f00-54a1-4809-9856-efb0ee40bd24-serviceca\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:39.695606 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695528    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-system-cni-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.695606 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695563    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-binary-copy\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.695606 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.695587    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.698046 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.698025    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:39.698141 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.698112    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:09:39.700702 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.700680    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:09:39.703396 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.703377    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.703498 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:39.703428    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2677e9e089ae4c3ce22398c86c934d7d.slice/crio-54997c0f21445fa127f7c31027a61b9a9c73aeb4b577450da384c4c885c5d6f7 WatchSource:0}: Error finding container 54997c0f21445fa127f7c31027a61b9a9c73aeb4b577450da384c4c885c5d6f7: Status 404 returned error can't find the container with id 54997c0f21445fa127f7c31027a61b9a9c73aeb4b577450da384c4c885c5d6f7
Nov 03 16:09:39.705910 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.705891    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:39.708590 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.708550    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.715891 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.715839    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal" event={"ID":"2677e9e089ae4c3ce22398c86c934d7d","Type":"ContainerStarted","Data":"54997c0f21445fa127f7c31027a61b9a9c73aeb4b577450da384c4c885c5d6f7"}
Nov 03 16:09:39.717387 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.717366    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal" event={"ID":"608951668150694fd6003da288232820","Type":"ContainerStarted","Data":"987d64b915ab5b9587adb18f4ec8af3d3d9179036bd21512ceebd2aa91a707fd"}
Nov 03 16:09:39.742690 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.742668    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 16:09:39.742770 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.742693    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 16:09:39.742770 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.742740    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 16:09:39.742877 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.742855    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 16:09:39.743130 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.743104    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 16:09:39.743130 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.743118    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 16:09:39.743306 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.743149    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 16:09:39.743306 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.743229    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 16:09:39.743398 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.743375    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kkrgk\""
Nov 03 16:09:39.743486 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.743433    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tk7qx\""
Nov 03 16:09:39.746129 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.746116    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 16:09:39.746189 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.746169    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 16:09:39.746189 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.746178    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9gv4t\""
Nov 03 16:09:39.748238 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.748226    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 16:09:39.748486 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.748473    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 16:09:39.748563 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.748551    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4xv66\""
Nov 03 16:09:39.753947 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.753893    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x72k5\""
Nov 03 16:09:39.753947 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.753903    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 16:09:39.753947 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.753929    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 16:09:39.754086 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.753992    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 16:09:39.759444 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.759424    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 16:09:39.759517 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.759427    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-4vrm9\""
Nov 03 16:09:39.759517 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.759484    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 16:09:39.761398 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.761383    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 16:09:39.761857 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.761841    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-xzbs4\""
Nov 03 16:09:39.764259 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.764245    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 16:09:39.785405 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.785378    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 16:09:39.786921 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.786906    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xhvl5\""
Nov 03 16:09:39.793099 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.793084    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 16:09:39.793673 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.793658    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 16:09:39.793731 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.793702    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sxk2h\""
Nov 03 16:09:39.796111 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796096    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-multus-daemon-config\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.796162 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796119    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-multus\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.796162 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796139    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.796162 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796155    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-bin\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.796253 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796171    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.796253 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796228    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-binary-copy\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.796328 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.796328 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796275    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.796328 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796297    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-k8s-cni-cncf-io\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.796439 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796409    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-netns\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.796439 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796432    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c14a9e56-182c-4abc-aa30-84b7958f775d-iptables-alerter-script\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:39.796522 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796509    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-node-log\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.796577 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796560    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-socket-dir-parent\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.796651 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796632    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-etc-kubernetes\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.796705 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796688    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7l95\" (UniqueName: \"kubernetes.io/projected/1c560131-5ae7-42ef-975b-23af6909fb89-kube-api-access-c7l95\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.796747 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796721    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-var-lib-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.796795 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796780    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-log-socket\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.796866 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796803    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:39.796866 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgqtp\" (UniqueName: \"kubernetes.io/projected/cffd0b63-ba73-433f-886f-fcae960b41a1-kube-api-access-sgqtp\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:39.796866 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796841    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4pnj\" (UniqueName: \"kubernetes.io/projected/1897a95d-8237-4a69-9ae8-c229490ba88c-kube-api-access-k4pnj\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.797004 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796883    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-kubelet\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.797004 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-slash\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.797004 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796955    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-config\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.797004 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.796982    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-env-overrides\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.797173 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797004    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-socket-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.797173 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:39.797173 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797052    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-modprobe-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.797173 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797067    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0599707f-1cdb-4e9f-99da-72703b028198-hosts-file\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:39.797173 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797083    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.797173 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797128    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-kubernetes\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.797441 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797189    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-ovn\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.797441 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797235    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.797441 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797234    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-dbus\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:39.797441 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797305    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-sys\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.797441 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-lib-modules\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.797441 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797397    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjvvq\" (UniqueName: \"kubernetes.io/projected/d80b9f00-54a1-4809-9856-efb0ee40bd24-kube-api-access-sjvvq\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:39.797441 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797419    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-conf-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.797441 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797434    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-systemd-units\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797450    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797470    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysconfig\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797490    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797525    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-conf\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797556    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-system-cni-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797569    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-binary-copy\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797585    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl446\" (UniqueName: \"kubernetes.io/projected/dadfa10c-198c-4f93-869f-377048834934-kube-api-access-pl446\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797572    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797611    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-system-cni-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797617    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-bin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797651    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-netd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797697    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-tmp\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.797768 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797795    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797812    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-cnibin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797840    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1897a95d-8237-4a69-9ae8-c229490ba88c-ovn-node-metrics-cert\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797879    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfl52\" (UniqueName: \"kubernetes.io/projected/38701b60-1908-4568-81ed-0c5a91716678-kube-api-access-sfl52\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797913    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2cfc5077-0c47-48fd-a247-51156f01935c-agent-certs\") pod \"konnectivity-agent-pkn6d\" (UID: \"2cfc5077-0c47-48fd-a247-51156f01935c\") " pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797952    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d80b9f00-54a1-4809-9856-efb0ee40bd24-serviceca\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.797986    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4brmn\" (UniqueName: \"kubernetes.io/projected/c14a9e56-182c-4abc-aa30-84b7958f775d-kube-api-access-4brmn\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-etc-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-sys-fs\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798066    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw9c7\" (UniqueName: \"kubernetes.io/projected/f239d145-6b28-421b-a024-e86135e1fb4d-kube-api-access-kw9c7\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798092    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7nln\" (UniqueName: \"kubernetes.io/projected/0599707f-1cdb-4e9f-99da-72703b028198-kube-api-access-r7nln\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798118    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d80b9f00-54a1-4809-9856-efb0ee40bd24-host\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798148    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c14a9e56-182c-4abc-aa30-84b7958f775d-host-slash\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798154    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798175    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-kubelet-config\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798199    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798223    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798231    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 16:09:39.798358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798237    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798240    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d80b9f00-54a1-4809-9856-efb0ee40bd24-host\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798284    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-systemd\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798297    2582 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798310    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-device-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798323    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798334    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-run\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798338    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vdn42"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798341    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0599707f-1cdb-4e9f-99da-72703b028198-tmp-dir\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798384    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d80b9f00-54a1-4809-9856-efb0ee40bd24-serviceca\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-cnibin\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-cni-binary-copy\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798483    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-cnibin\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798488    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-hostroot\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798512    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-host\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798573    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-multus-certs\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-kubelet\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.799097 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798629    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-script-lib\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.799670 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798656    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-etc-tuned\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.799670 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798694    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-netns\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.799670 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798726    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-system-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.799670 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798774    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-systemd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.799670 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798794    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.799670 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798811    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-registration-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.799670 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798829    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-var-lib-kubelet\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.799670 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.798856    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2cfc5077-0c47-48fd-a247-51156f01935c-konnectivity-ca\") pod \"konnectivity-agent-pkn6d\" (UID: \"2cfc5077-0c47-48fd-a247-51156f01935c\") " pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:09:39.860686 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.860659    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 16:09:39.860686 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.860682    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 16:09:39.860845 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.860693    2582 projected.go:196] Error preparing data for projected volume kube-api-access-zsxr9 for pod openshift-network-diagnostics/network-check-target-lqvng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:39.860845 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.860777    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9 podName:a67334f9-d13e-452c-b266-e0e996dd7db3 nodeName:}" failed. No retries permitted until 2026-11-03 16:09:40.36074058 +0000 UTC m=+3.193381252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zsxr9" (UniqueName: "kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9") pod "network-check-target-lqvng" (UID: "a67334f9-d13e-452c-b266-e0e996dd7db3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:39.860948 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.860929    2582 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 16:09:39.864536 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.864514    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl446\" (UniqueName: \"kubernetes.io/projected/dadfa10c-198c-4f93-869f-377048834934-kube-api-access-pl446\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:39.864536 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.864536    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjvvq\" (UniqueName: \"kubernetes.io/projected/d80b9f00-54a1-4809-9856-efb0ee40bd24-kube-api-access-sjvvq\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:39.899698 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899675    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-system-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.899820 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899705    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-systemd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.899820 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899725    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.899820 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899768    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-registration-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.899820 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899794    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-var-lib-kubelet\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.899820 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899796    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-system-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.899820 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899795    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-systemd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899834    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899840    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2cfc5077-0c47-48fd-a247-51156f01935c-konnectivity-ca\") pod \"konnectivity-agent-pkn6d\" (UID: \"2cfc5077-0c47-48fd-a247-51156f01935c\") " pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899853    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-var-lib-kubelet\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899868    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-multus-daemon-config\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899897    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-registration-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899918    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-multus\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899934    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-bin\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899974    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-multus\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899982    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.899997    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900021    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-k8s-cni-cncf-io\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900027    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900027    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-bin\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900048    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-netns\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900075    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c14a9e56-182c-4abc-aa30-84b7958f775d-iptables-alerter-script\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:39.900114 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900080    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-k8s-cni-cncf-io\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900108    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-node-log\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900137    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-netns\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-socket-dir-parent\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900179    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-etc-kubernetes\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900192    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-node-log\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900218    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7l95\" (UniqueName: \"kubernetes.io/projected/1c560131-5ae7-42ef-975b-23af6909fb89-kube-api-access-c7l95\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900237    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-etc-kubernetes\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-var-lib-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900268    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-socket-dir-parent\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-log-socket\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900330    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgqtp\" (UniqueName: \"kubernetes.io/projected/cffd0b63-ba73-433f-886f-fcae960b41a1-kube-api-access-sgqtp\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900336    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-log-socket\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900307    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-var-lib-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900356    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pnj\" (UniqueName: \"kubernetes.io/projected/1897a95d-8237-4a69-9ae8-c229490ba88c-kube-api-access-k4pnj\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900385    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-kubelet\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900408    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-slash\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.900920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900432    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-config\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900458    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-env-overrides\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900464    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-kubelet\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900486    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-socket-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900503    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-multus-daemon-config\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900513    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900515    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2cfc5077-0c47-48fd-a247-51156f01935c-konnectivity-ca\") pod \"konnectivity-agent-pkn6d\" (UID: \"2cfc5077-0c47-48fd-a247-51156f01935c\") " pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900539    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-modprobe-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900637    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-slash\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900645    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-modprobe-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900650    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-socket-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.900662    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900679    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c14a9e56-182c-4abc-aa30-84b7958f775d-iptables-alerter-script\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900688    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0599707f-1cdb-4e9f-99da-72703b028198-hosts-file\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900720    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900746    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-kubernetes\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900749    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0599707f-1cdb-4e9f-99da-72703b028198-hosts-file\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:39.901657 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.900760    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret podName:c6f6992c-48b9-4bb5-9b85-a37552b64c8b nodeName:}" failed. No retries permitted until 2026-11-03 16:09:40.400744028 +0000 UTC m=+3.233384680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret") pod "global-pull-secret-syncer-tbpr9" (UID: "c6f6992c-48b9-4bb5-9b85-a37552b64c8b") : object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900781    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-kubernetes\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900802    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900809    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-ovn\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900832    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-dbus\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900847    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-sys\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900846    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-ovn\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900881    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-lib-modules\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900908    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-sys\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-conf-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900932    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-conf-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-systemd-units\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900955    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-dbus\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.900985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901000    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-systemd-units\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901010    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901017    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysconfig\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901024    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-lib-modules\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.902751 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901041    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901039    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-env-overrides\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-conf\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901057    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-config\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901063    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysconfig\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901087    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-bin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-netd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901115    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-bin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901127    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-tmp\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901131    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-conf\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901148    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-netd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901153    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901176    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901189    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-cnibin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.901216    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901250    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-cnibin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1897a95d-8237-4a69-9ae8-c229490ba88c-ovn-node-metrics-cert\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.903685 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:39.901278    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs podName:cffd0b63-ba73-433f-886f-fcae960b41a1 nodeName:}" failed. No retries permitted until 2026-11-03 16:09:40.401247442 +0000 UTC m=+3.233888110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs") pod "network-metrics-daemon-k5rqd" (UID: "cffd0b63-ba73-433f-886f-fcae960b41a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901317    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfl52\" (UniqueName: \"kubernetes.io/projected/38701b60-1908-4568-81ed-0c5a91716678-kube-api-access-sfl52\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901347    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2cfc5077-0c47-48fd-a247-51156f01935c-agent-certs\") pod \"konnectivity-agent-pkn6d\" (UID: \"2cfc5077-0c47-48fd-a247-51156f01935c\") " pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4brmn\" (UniqueName: \"kubernetes.io/projected/c14a9e56-182c-4abc-aa30-84b7958f775d-kube-api-access-4brmn\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901399    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-etc-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901433    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-etc-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-sys-fs\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901521    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw9c7\" (UniqueName: \"kubernetes.io/projected/f239d145-6b28-421b-a024-e86135e1fb4d-kube-api-access-kw9c7\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901548    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7nln\" (UniqueName: \"kubernetes.io/projected/0599707f-1cdb-4e9f-99da-72703b028198-kube-api-access-r7nln\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901554    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-sys-fs\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901575    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c14a9e56-182c-4abc-aa30-84b7958f775d-host-slash\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-kubelet-config\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-systemd\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901639    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c14a9e56-182c-4abc-aa30-84b7958f775d-host-slash\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-device-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901674    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-run\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0599707f-1cdb-4e9f-99da-72703b028198-tmp-dir\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:39.904273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901756    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-device-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901813    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-kubelet-config\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901913    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-systemd\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901934    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-cni-binary-copy\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901960    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-hostroot\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901829    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-run\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902022    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-host\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.901977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-host\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902047    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-hostroot\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902074    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-multus-certs\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902103    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-kubelet\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902120    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-multus-certs\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902130    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-script-lib\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902169    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-kubelet\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902218    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-etc-tuned\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902248    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-netns\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902307    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-netns\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.902739    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0599707f-1cdb-4e9f-99da-72703b028198-tmp-dir\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:39.904758 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.903029    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-script-lib\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.905293 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.903032    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-cni-binary-copy\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.905293 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.903618    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-tmp\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.905293 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.903751    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1897a95d-8237-4a69-9ae8-c229490ba88c-ovn-node-metrics-cert\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.905293 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.904069    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2cfc5077-0c47-48fd-a247-51156f01935c-agent-certs\") pod \"konnectivity-agent-pkn6d\" (UID: \"2cfc5077-0c47-48fd-a247-51156f01935c\") " pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:09:39.905293 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.904608    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-etc-tuned\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.967284 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.967254    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4brmn\" (UniqueName: \"kubernetes.io/projected/c14a9e56-182c-4abc-aa30-84b7958f775d-kube-api-access-4brmn\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:39.967412 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.967306    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7nln\" (UniqueName: \"kubernetes.io/projected/0599707f-1cdb-4e9f-99da-72703b028198-kube-api-access-r7nln\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:39.967412 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.967316    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgqtp\" (UniqueName: \"kubernetes.io/projected/cffd0b63-ba73-433f-886f-fcae960b41a1-kube-api-access-sgqtp\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:39.967412 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.967319    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7l95\" (UniqueName: \"kubernetes.io/projected/1c560131-5ae7-42ef-975b-23af6909fb89-kube-api-access-c7l95\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:39.967412 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.967318    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfl52\" (UniqueName: \"kubernetes.io/projected/38701b60-1908-4568-81ed-0c5a91716678-kube-api-access-sfl52\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:39.967412 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.967329    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4pnj\" (UniqueName: \"kubernetes.io/projected/1897a95d-8237-4a69-9ae8-c229490ba88c-kube-api-access-k4pnj\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:39.967412 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.967332    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw9c7\" (UniqueName: \"kubernetes.io/projected/f239d145-6b28-421b-a024-e86135e1fb4d-kube-api-access-kw9c7\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:39.995372 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:39.995349    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:09:40.005648 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:40.005592    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd80b9f00_54a1_4809_9856_efb0ee40bd24.slice/crio-6a15c0e001c60f152152a613b3c0365ac1f8c5032b7af7caadb111e177f8929c WatchSource:0}: Error finding container 6a15c0e001c60f152152a613b3c0365ac1f8c5032b7af7caadb111e177f8929c: Status 404 returned error can't find the container with id 6a15c0e001c60f152152a613b3c0365ac1f8c5032b7af7caadb111e177f8929c
Nov 03 16:09:40.014115 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.014095    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:09:40.018194 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.018178    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:09:40.020687 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.020658    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:09:40.024611 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:40.024592    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38701b60_1908_4568_81ed_0c5a91716678.slice/crio-640504f20f815af422689f0cc264f247250f66c577fc07e82d003ed25f0e128e WatchSource:0}: Error finding container 640504f20f815af422689f0cc264f247250f66c577fc07e82d003ed25f0e128e: Status 404 returned error can't find the container with id 640504f20f815af422689f0cc264f247250f66c577fc07e82d003ed25f0e128e
Nov 03 16:09:40.028387 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.028367    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:09:40.029533 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:40.029508    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0599707f_1cdb_4e9f_99da_72703b028198.slice/crio-f78dc792b42754170fd76c386076ebb557d32b1e906205156736d45231680942 WatchSource:0}: Error finding container f78dc792b42754170fd76c386076ebb557d32b1e906205156736d45231680942: Status 404 returned error can't find the container with id f78dc792b42754170fd76c386076ebb557d32b1e906205156736d45231680942
Nov 03 16:09:40.030077 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.030059    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:09:40.031342 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:40.031323    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddadfa10c_198c_4f93_869f_377048834934.slice/crio-2e2a0550fd2598e68d0ca1f4731c713f02134f627850a60044638e380f61fc2b WatchSource:0}: Error finding container 2e2a0550fd2598e68d0ca1f4731c713f02134f627850a60044638e380f61fc2b: Status 404 returned error can't find the container with id 2e2a0550fd2598e68d0ca1f4731c713f02134f627850a60044638e380f61fc2b
Nov 03 16:09:40.034752 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.034434    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qgfrq"
Nov 03 16:09:40.034752 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.034644    2582 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 16:09:40.040980 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.040962    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:09:40.044102 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:40.044079    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf239d145_6b28_421b_a024_e86135e1fb4d.slice/crio-4d67fc0bee521b1a0c4198f96b87ad9bc5ceea29046fbb617ad8e4076d91d853 WatchSource:0}: Error finding container 4d67fc0bee521b1a0c4198f96b87ad9bc5ceea29046fbb617ad8e4076d91d853: Status 404 returned error can't find the container with id 4d67fc0bee521b1a0c4198f96b87ad9bc5ceea29046fbb617ad8e4076d91d853
Nov 03 16:09:40.046444 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:40.046413    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cfc5077_0c47_48fd_a247_51156f01935c.slice/crio-52fa4ae03cad2bfea8518e5e02e5c89a84500eaf6176c3682eabf55b08d4df93 WatchSource:0}: Error finding container 52fa4ae03cad2bfea8518e5e02e5c89a84500eaf6176c3682eabf55b08d4df93: Status 404 returned error can't find the container with id 52fa4ae03cad2bfea8518e5e02e5c89a84500eaf6176c3682eabf55b08d4df93
Nov 03 16:09:40.048656 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:40.048629    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c560131_5ae7_42ef_975b_23af6909fb89.slice/crio-29dad769d3401de44406bfe76091c3caa1c312098ec7c27d72c89c748eec01d8 WatchSource:0}: Error finding container 29dad769d3401de44406bfe76091c3caa1c312098ec7c27d72c89c748eec01d8: Status 404 returned error can't find the container with id 29dad769d3401de44406bfe76091c3caa1c312098ec7c27d72c89c748eec01d8
Nov 03 16:09:40.053290 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:40.053267    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc14a9e56_182c_4abc_aa30_84b7958f775d.slice/crio-92821c6fc58b809f6cd483310cd470578eb71531fc5f8dce6914f49fa9b015c1 WatchSource:0}: Error finding container 92821c6fc58b809f6cd483310cd470578eb71531fc5f8dce6914f49fa9b015c1: Status 404 returned error can't find the container with id 92821c6fc58b809f6cd483310cd470578eb71531fc5f8dce6914f49fa9b015c1
Nov 03 16:09:40.061590 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.061569    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:09:40.070929 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:09:40.070909    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1897a95d_8237_4a69_9ae8_c229490ba88c.slice/crio-f88d78bcdab59ab026d9e8b15d0f7dbd94b3b631627ecd7576c5dcbb205ab3f6 WatchSource:0}: Error finding container f88d78bcdab59ab026d9e8b15d0f7dbd94b3b631627ecd7576c5dcbb205ab3f6: Status 404 returned error can't find the container with id f88d78bcdab59ab026d9e8b15d0f7dbd94b3b631627ecd7576c5dcbb205ab3f6
Nov 03 16:09:40.405280 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.405181    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:40.405280 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.405238    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:40.405507 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.405292    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:40.405507 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:40.405336    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:40.405507 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:40.405428    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs podName:cffd0b63-ba73-433f-886f-fcae960b41a1 nodeName:}" failed. No retries permitted until 2026-11-03 16:09:41.405407927 +0000 UTC m=+4.238048579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs") pod "network-metrics-daemon-k5rqd" (UID: "cffd0b63-ba73-433f-886f-fcae960b41a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:40.405507 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:40.405427    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:40.405507 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:40.405452    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 16:09:40.405507 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:40.405473    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 16:09:40.405507 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:40.405487    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret podName:c6f6992c-48b9-4bb5-9b85-a37552b64c8b nodeName:}" failed. No retries permitted until 2026-11-03 16:09:41.405474376 +0000 UTC m=+4.238115042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret") pod "global-pull-secret-syncer-tbpr9" (UID: "c6f6992c-48b9-4bb5-9b85-a37552b64c8b") : object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:40.405507 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:40.405487    2582 projected.go:196] Error preparing data for projected volume kube-api-access-zsxr9 for pod openshift-network-diagnostics/network-check-target-lqvng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:40.405817 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:40.405537    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9 podName:a67334f9-d13e-452c-b266-e0e996dd7db3 nodeName:}" failed. No retries permitted until 2026-11-03 16:09:41.405522424 +0000 UTC m=+4.238163099 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zsxr9" (UniqueName: "kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9") pod "network-check-target-lqvng" (UID: "a67334f9-d13e-452c-b266-e0e996dd7db3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:40.711536 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.711467    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:40.712175 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:40.711574    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:40.720622 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.720585    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gfrfv" event={"ID":"d80b9f00-54a1-4809-9856-efb0ee40bd24","Type":"ContainerStarted","Data":"6a15c0e001c60f152152a613b3c0365ac1f8c5032b7af7caadb111e177f8929c"}
Nov 03 16:09:40.722067 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.722033    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l" event={"ID":"1897a95d-8237-4a69-9ae8-c229490ba88c","Type":"ContainerStarted","Data":"f88d78bcdab59ab026d9e8b15d0f7dbd94b3b631627ecd7576c5dcbb205ab3f6"}
Nov 03 16:09:40.723155 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.723133    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-864tf" event={"ID":"c14a9e56-182c-4abc-aa30-84b7958f775d","Type":"ContainerStarted","Data":"92821c6fc58b809f6cd483310cd470578eb71531fc5f8dce6914f49fa9b015c1"}
Nov 03 16:09:40.724070 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.724049    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qgfrq" event={"ID":"1c560131-5ae7-42ef-975b-23af6909fb89","Type":"ContainerStarted","Data":"29dad769d3401de44406bfe76091c3caa1c312098ec7c27d72c89c748eec01d8"}
Nov 03 16:09:40.725082 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.725061    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pkn6d" event={"ID":"2cfc5077-0c47-48fd-a247-51156f01935c","Type":"ContainerStarted","Data":"52fa4ae03cad2bfea8518e5e02e5c89a84500eaf6176c3682eabf55b08d4df93"}
Nov 03 16:09:40.726286 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.726263    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5" event={"ID":"f239d145-6b28-421b-a024-e86135e1fb4d","Type":"ContainerStarted","Data":"4d67fc0bee521b1a0c4198f96b87ad9bc5ceea29046fbb617ad8e4076d91d853"}
Nov 03 16:09:40.727578 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.727553    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g2fnp" event={"ID":"dadfa10c-198c-4f93-869f-377048834934","Type":"ContainerStarted","Data":"2e2a0550fd2598e68d0ca1f4731c713f02134f627850a60044638e380f61fc2b"}
Nov 03 16:09:40.728766 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.728722    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j8mcw" event={"ID":"0599707f-1cdb-4e9f-99da-72703b028198","Type":"ContainerStarted","Data":"f78dc792b42754170fd76c386076ebb557d32b1e906205156736d45231680942"}
Nov 03 16:09:40.729809 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.729789    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8k9vh" event={"ID":"38701b60-1908-4568-81ed-0c5a91716678","Type":"ContainerStarted","Data":"640504f20f815af422689f0cc264f247250f66c577fc07e82d003ed25f0e128e"}
Nov 03 16:09:40.802852 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.802800    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 16:04:40 +0000 UTC" deadline="2028-06-04 04:47:25.531411274 +0000 UTC"
Nov 03 16:09:40.802852 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:40.802843    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13884h37m44.728574401s"
Nov 03 16:09:41.415348 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:41.415303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:41.415537 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:41.415389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:41.415537 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:41.415449    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:41.415643 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:41.415572    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:41.415643 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:41.415636    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs podName:cffd0b63-ba73-433f-886f-fcae960b41a1 nodeName:}" failed. No retries permitted until 2026-11-03 16:09:43.4156194 +0000 UTC m=+6.248260076 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs") pod "network-metrics-daemon-k5rqd" (UID: "cffd0b63-ba73-433f-886f-fcae960b41a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:41.415743 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:41.415715    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:41.415850 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:41.415826    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret podName:c6f6992c-48b9-4bb5-9b85-a37552b64c8b nodeName:}" failed. No retries permitted until 2026-11-03 16:09:43.415787874 +0000 UTC m=+6.248428546 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret") pod "global-pull-secret-syncer-tbpr9" (UID: "c6f6992c-48b9-4bb5-9b85-a37552b64c8b") : object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:41.416019 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:41.416003    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 16:09:41.416139 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:41.416127    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 16:09:41.416227 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:41.416145    2582 projected.go:196] Error preparing data for projected volume kube-api-access-zsxr9 for pod openshift-network-diagnostics/network-check-target-lqvng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:41.416227 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:41.416198    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9 podName:a67334f9-d13e-452c-b266-e0e996dd7db3 nodeName:}" failed. No retries permitted until 2026-11-03 16:09:43.416177883 +0000 UTC m=+6.248818567 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zsxr9" (UniqueName: "kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9") pod "network-check-target-lqvng" (UID: "a67334f9-d13e-452c-b266-e0e996dd7db3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:41.713307 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:41.712962    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:41.713307 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:41.713118    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:41.713307 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:41.713161    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:41.713307 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:41.713234    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:09:41.803366 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:41.803293    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 16:04:40 +0000 UTC" deadline="2028-06-06 23:49:20.626568278 +0000 UTC"
Nov 03 16:09:41.803366 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:41.803363    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13951h39m38.823209368s"
Nov 03 16:09:42.712357 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:42.712299    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:42.712528 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:42.712470    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:43.433477 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:43.433440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:43.433912 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:43.433492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:43.433912 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:43.433515    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:43.433912 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:43.433609    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:43.433912 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:43.433612    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:43.433912 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:43.433627    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 16:09:43.433912 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:43.433647    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 16:09:43.433912 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:43.433660    2582 projected.go:196] Error preparing data for projected volume kube-api-access-zsxr9 for pod openshift-network-diagnostics/network-check-target-lqvng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:43.433912 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:43.433669    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret podName:c6f6992c-48b9-4bb5-9b85-a37552b64c8b nodeName:}" failed. No retries permitted until 2026-11-03 16:09:47.433656149 +0000 UTC m=+10.266296817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret") pod "global-pull-secret-syncer-tbpr9" (UID: "c6f6992c-48b9-4bb5-9b85-a37552b64c8b") : object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:43.433912 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:43.433684    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs podName:cffd0b63-ba73-433f-886f-fcae960b41a1 nodeName:}" failed. No retries permitted until 2026-11-03 16:09:47.433677716 +0000 UTC m=+10.266318367 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs") pod "network-metrics-daemon-k5rqd" (UID: "cffd0b63-ba73-433f-886f-fcae960b41a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:43.433912 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:43.433699    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9 podName:a67334f9-d13e-452c-b266-e0e996dd7db3 nodeName:}" failed. No retries permitted until 2026-11-03 16:09:47.433687826 +0000 UTC m=+10.266328518 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zsxr9" (UniqueName: "kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9") pod "network-check-target-lqvng" (UID: "a67334f9-d13e-452c-b266-e0e996dd7db3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:43.711614 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:43.711501    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:43.711614 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:43.711505    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:43.711850 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:43.711636    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:43.711850 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:43.711722    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:09:43.747375 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:43.747338    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal" event={"ID":"608951668150694fd6003da288232820","Type":"ContainerStarted","Data":"e8330244d709ef9b9882a3140eebbd0ab3c4c38a8b166d504cb9e598af6aeb95"}
Nov 03 16:09:43.748850 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:43.748825    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pkn6d" event={"ID":"2cfc5077-0c47-48fd-a247-51156f01935c","Type":"ContainerStarted","Data":"54da03e718f758846445398a37dc50175f5a8a71b4e0097b8c5243e01d6d9bda"}
Nov 03 16:09:44.006711 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:44.006609    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pkn6d" podStartSLOduration=3.490697579 podStartE2EDuration="6.006592657s" totalImagesPullingTime="2.515895078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:09:44.0063171 +0000 UTC m=+6.838957773" watchObservedRunningTime="2026-11-03 16:09:44.006592657 +0000 UTC m=+6.839233329"
Nov 03 16:09:44.006907 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:44.006715    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal" podStartSLOduration=5.006707546 podStartE2EDuration="5.006707546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:09:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:09:43.877695971 +0000 UTC m=+6.710336647" watchObservedRunningTime="2026-11-03 16:09:44.006707546 +0000 UTC m=+6.839348221"
Nov 03 16:09:44.712457 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:44.712424    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:44.713029 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:44.712581    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:45.645964 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:45.645920    2582 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:09:45.646658 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:45.646626    2582 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:09:45.712780 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:45.712202    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:45.712780 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:45.712351    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:45.712780 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:45.712642    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:45.712780 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:45.712743    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:09:45.753206 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:45.753175    2582 generic.go:356] "Generic (PLEG): container finished" podID="2677e9e089ae4c3ce22398c86c934d7d" containerID="0111f5c972adbd997ebaf61a26a04445e32592d1bd1fff16c4f40b031215ded2" exitCode=0
Nov 03 16:09:45.753369 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:45.753259    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal" event={"ID":"2677e9e089ae4c3ce22398c86c934d7d","Type":"ContainerDied","Data":"0111f5c972adbd997ebaf61a26a04445e32592d1bd1fff16c4f40b031215ded2"}
Nov 03 16:09:46.711932 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:46.711852    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:46.712096 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:46.711991    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:46.758277 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:46.758240    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal" event={"ID":"2677e9e089ae4c3ce22398c86c934d7d","Type":"ContainerStarted","Data":"dc48f1009cc4cf1d6291e136430da1ffe1c43a34b96d8de362789f78c063542b"}
Nov 03 16:09:46.758277 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:46.758276    2582 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 16:09:46.877387 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:46.877330    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal" podStartSLOduration=7.877315052 podStartE2EDuration="7.877315052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:09:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:09:46.877165183 +0000 UTC m=+9.709805856" watchObservedRunningTime="2026-11-03 16:09:46.877315052 +0000 UTC m=+9.709955722"
Nov 03 16:09:47.469329 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:47.469256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:47.469483 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:47.469342    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:47.469483 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:47.469393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:47.469593 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:47.469522    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:47.469593 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:47.469586    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs podName:cffd0b63-ba73-433f-886f-fcae960b41a1 nodeName:}" failed. No retries permitted until 2026-11-03 16:09:55.469565851 +0000 UTC m=+18.302206522 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs") pod "network-metrics-daemon-k5rqd" (UID: "cffd0b63-ba73-433f-886f-fcae960b41a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:47.469892 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:47.469853    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 16:09:47.469892 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:47.469885    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 16:09:47.469892 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:47.469899    2582 projected.go:196] Error preparing data for projected volume kube-api-access-zsxr9 for pod openshift-network-diagnostics/network-check-target-lqvng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:47.470178 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:47.469949    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9 podName:a67334f9-d13e-452c-b266-e0e996dd7db3 nodeName:}" failed. No retries permitted until 2026-11-03 16:09:55.46993171 +0000 UTC m=+18.302572375 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zsxr9" (UniqueName: "kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9") pod "network-check-target-lqvng" (UID: "a67334f9-d13e-452c-b266-e0e996dd7db3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:47.470178 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:47.470018    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:47.470178 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:47.470058    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret podName:c6f6992c-48b9-4bb5-9b85-a37552b64c8b nodeName:}" failed. No retries permitted until 2026-11-03 16:09:55.47004546 +0000 UTC m=+18.302686124 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret") pod "global-pull-secret-syncer-tbpr9" (UID: "c6f6992c-48b9-4bb5-9b85-a37552b64c8b") : object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:47.713185 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:47.713148    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:47.713345 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:47.713283    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:47.713779 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:47.713758    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:47.715093 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:47.715059    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:09:47.761989 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:47.761922    2582 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="7ca75a9c01edfbc25b850c51bd02c7f20879c88ce2f2f8f1dd867c36a70b9478" exitCode=0
Nov 03 16:09:47.762434 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:47.762010    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g2fnp" event={"ID":"dadfa10c-198c-4f93-869f-377048834934","Type":"ContainerDied","Data":"7ca75a9c01edfbc25b850c51bd02c7f20879c88ce2f2f8f1dd867c36a70b9478"}
Nov 03 16:09:48.711909 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:48.711874    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:48.712099 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:48.711998    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:48.766498 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:48.766449    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5" event={"ID":"f239d145-6b28-421b-a024-e86135e1fb4d","Type":"ContainerStarted","Data":"2db4665edf3fe61127063ed2896c31f06b239b222f2097446cb2eebe472524be"}
Nov 03 16:09:49.711921 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:49.711843    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:49.712078 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:49.711982    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:49.712078 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:49.712011    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:49.712225 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:49.712083    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:09:49.770492 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:49.770453    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qgfrq" event={"ID":"1c560131-5ae7-42ef-975b-23af6909fb89","Type":"ContainerStarted","Data":"b4f769c479f8a101ebab235dce7f85580ec680ba9c88955be5d8b1414879ab17"}
Nov 03 16:09:49.772240 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:49.772214    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j8mcw" event={"ID":"0599707f-1cdb-4e9f-99da-72703b028198","Type":"ContainerStarted","Data":"53ef3b59f5a313526714da8d1165639ca4f634c92cc0d51929228a771721da5f"}
Nov 03 16:09:49.912300 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:49.912246    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-qgfrq" podStartSLOduration=2.364407891 podStartE2EDuration="11.912233278s" totalImagesPullingTime="9.547825387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:09:49.911930566 +0000 UTC m=+12.744571238" watchObservedRunningTime="2026-11-03 16:09:49.912233278 +0000 UTC m=+12.744873950"
Nov 03 16:09:50.034108 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:50.033994    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-j8mcw" podStartSLOduration=2.602925103 podStartE2EDuration="12.033973535s" totalImagesPullingTime="9.431048432s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:09:50.033715763 +0000 UTC m=+12.866356437" watchObservedRunningTime="2026-11-03 16:09:50.033973535 +0000 UTC m=+12.866614208"
Nov 03 16:09:50.712120 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:50.712036    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:50.712292 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:50.712171    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:50.775465 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:50.775425    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gfrfv" event={"ID":"d80b9f00-54a1-4809-9856-efb0ee40bd24","Type":"ContainerStarted","Data":"dcce5bc48a256a2d28c81d6dd5d83c54814b0c92a1b0871280d9e074c0edf2aa"}
Nov 03 16:09:50.895512 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:50.895455    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gfrfv" podStartSLOduration=2.555225886 podStartE2EDuration="12.895441128s" totalImagesPullingTime="10.340215242s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:09:50.894650577 +0000 UTC m=+13.727291250" watchObservedRunningTime="2026-11-03 16:09:50.895441128 +0000 UTC m=+13.728081800"
Nov 03 16:09:51.712902 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:51.712764    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:51.713103 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:51.712896    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:09:51.713296 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:51.713273    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:51.713429 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:51.713385    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:51.956869 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:51.956707    2582 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 16:09:52.669927 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:52.669832    2582 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T16:09:51.956865304Z","UUID":"a6d2a1ab-08b6-446c-bd99-18c0cf8c17aa","Handler":null,"Name":"","Endpoint":""}
Nov 03 16:09:52.671162 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:52.671140    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 16:09:52.671162 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:52.671165    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 16:09:52.712400 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:52.712372    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:52.712512 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:52.712472    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:52.780139 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:52.780110    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5" event={"ID":"f239d145-6b28-421b-a024-e86135e1fb4d","Type":"ContainerStarted","Data":"d241c4a95617722f39b0960f8b61f59ccd2682853ac438f2398395206ba5b470"}
Nov 03 16:09:52.781527 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:52.781498    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-864tf" event={"ID":"c14a9e56-182c-4abc-aa30-84b7958f775d","Type":"ContainerStarted","Data":"a3f0049c6cf183e2f414655f4cecb68400333694cb56291023ee6e3df0063dbf"}
Nov 03 16:09:52.901972 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:52.901884    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-864tf" podStartSLOduration=3.5242271499999998 podStartE2EDuration="14.901871678s" totalImagesPullingTime="11.377644528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:09:52.901629327 +0000 UTC m=+15.734269991" watchObservedRunningTime="2026-11-03 16:09:52.901871678 +0000 UTC m=+15.734512349"
Nov 03 16:09:53.711852 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:53.711817    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:53.712229 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:53.711934    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:53.712229 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:53.711987    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:53.712229 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:53.712069    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:09:54.712190 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:54.712157    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:54.712702 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:54.712291    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:55.531497 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:55.531463    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:55.531637 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:55.531512    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:55.531637 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:55.531560    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:55.531637 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:55.531625    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:55.531766 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:55.531643    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:55.531766 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:55.531690    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret podName:c6f6992c-48b9-4bb5-9b85-a37552b64c8b nodeName:}" failed. No retries permitted until 2026-11-03 16:10:11.53167314 +0000 UTC m=+34.364313808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret") pod "global-pull-secret-syncer-tbpr9" (UID: "c6f6992c-48b9-4bb5-9b85-a37552b64c8b") : object "kube-system"/"original-pull-secret" not registered
Nov 03 16:09:55.531766 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:55.531703    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 16:09:55.531766 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:55.531708    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs podName:cffd0b63-ba73-433f-886f-fcae960b41a1 nodeName:}" failed. No retries permitted until 2026-11-03 16:10:11.531700155 +0000 UTC m=+34.364340808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs") pod "network-metrics-daemon-k5rqd" (UID: "cffd0b63-ba73-433f-886f-fcae960b41a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:09:55.531766 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:55.531721    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 16:09:55.531766 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:55.531751    2582 projected.go:196] Error preparing data for projected volume kube-api-access-zsxr9 for pod openshift-network-diagnostics/network-check-target-lqvng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:55.531942 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:55.531799    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9 podName:a67334f9-d13e-452c-b266-e0e996dd7db3 nodeName:}" failed. No retries permitted until 2026-11-03 16:10:11.531787638 +0000 UTC m=+34.364428289 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zsxr9" (UniqueName: "kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9") pod "network-check-target-lqvng" (UID: "a67334f9-d13e-452c-b266-e0e996dd7db3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:09:55.712394 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:55.712366    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:55.713021 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:55.712368    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:55.713021 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:55.712466    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:55.713021 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:55.712526    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:09:55.789142 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:55.789055    2582 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="06d0183b5d8b230e9fbf0cfa183c72580e22f714b7a35602e0f9e7532c2c1168" exitCode=0
Nov 03 16:09:55.789142 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:55.789105    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g2fnp" event={"ID":"dadfa10c-198c-4f93-869f-377048834934","Type":"ContainerDied","Data":"06d0183b5d8b230e9fbf0cfa183c72580e22f714b7a35602e0f9e7532c2c1168"}
Nov 03 16:09:56.712426 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:56.712393    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:56.712925 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:56.712519    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:57.712391 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:57.712359    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:57.712530 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:57.712450    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:57.712820 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:57.712542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:57.712820 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:57.712627    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:09:57.795557 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:57.795519    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5" event={"ID":"f239d145-6b28-421b-a024-e86135e1fb4d","Type":"ContainerStarted","Data":"ce0ac91feaa025009e51c78b4c5d3435016f95a7a8c14444605b597778ba585d"}
Nov 03 16:09:57.948945 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:57.948900    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5" podStartSLOduration=3.1543712839999998 podStartE2EDuration="19.948886636s" totalImagesPullingTime="16.794515352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:09:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:09:57.948630203 +0000 UTC m=+20.781270875" watchObservedRunningTime="2026-11-03 16:09:57.948886636 +0000 UTC m=+20.781527309"
Nov 03 16:09:58.712126 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:58.711929    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:09:58.712300 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:58.712210    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:09:58.799632 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:58.799594    2582 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="0d66c11959ae169f9b8dd0043755ffc4c9ce808663624cc847737e33304210c9" exitCode=0
Nov 03 16:09:58.800007 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:58.799685    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g2fnp" event={"ID":"dadfa10c-198c-4f93-869f-377048834934","Type":"ContainerDied","Data":"0d66c11959ae169f9b8dd0043755ffc4c9ce808663624cc847737e33304210c9"}
Nov 03 16:09:59.711745 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:59.711700    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:09:59.711913 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:09:59.711703    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:09:59.711913 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:59.711826    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:09:59.711913 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:09:59.711894    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:00.711669 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:00.711637    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:00.712091 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:00.711758    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:00.805148 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:00.805114    2582 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="56d89659b0c3462af1d9ffa49bcb2b2f8169a1c68dd96f43c7b06aa3b22c6b9f" exitCode=0
Nov 03 16:10:00.805314 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:00.805172    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g2fnp" event={"ID":"dadfa10c-198c-4f93-869f-377048834934","Type":"ContainerDied","Data":"56d89659b0c3462af1d9ffa49bcb2b2f8169a1c68dd96f43c7b06aa3b22c6b9f"}
Nov 03 16:10:00.931172 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:00.931144    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:10:00.931361 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:00.931260    2582 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 16:10:00.931753 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:00.931711    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:10:01.711610 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:01.711579    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:01.711610 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:01.711578    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:01.712085 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:01.711775    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:01.712085 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:01.711850    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:02.711605 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:02.711570    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:02.711865 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:02.711711    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:03.712140 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:03.712101    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:03.712615 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:03.712255    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:03.712741 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:03.712704    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:03.712848 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:03.712809    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:03.814298 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:03.814261    2582 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="3758f24798f703425eaf0fedaa6aa47fc3a5ddbc8a9492cd0f192436129d0bcc" exitCode=0
Nov 03 16:10:03.814298 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:03.814294    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g2fnp" event={"ID":"dadfa10c-198c-4f93-869f-377048834934","Type":"ContainerDied","Data":"3758f24798f703425eaf0fedaa6aa47fc3a5ddbc8a9492cd0f192436129d0bcc"}
Nov 03 16:10:04.711457 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:04.711416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:04.711624 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:04.711543    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:04.820343 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:04.820306    2582 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="54710dc6b002c87f18bbde44e603e48c14afb9be933290aac8fd90f2b9a7012a" exitCode=0
Nov 03 16:10:04.820792 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:04.820372    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g2fnp" event={"ID":"dadfa10c-198c-4f93-869f-377048834934","Type":"ContainerDied","Data":"54710dc6b002c87f18bbde44e603e48c14afb9be933290aac8fd90f2b9a7012a"}
Nov 03 16:10:05.712475 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:05.712409    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:05.712618 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:05.712409    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:05.712618 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:05.712522    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:05.712618 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:05.712574    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:05.829812 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:05.829775    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g2fnp" event={"ID":"dadfa10c-198c-4f93-869f-377048834934","Type":"ContainerStarted","Data":"7fd19a022503be755f89b157870570f69bd7a7732785a6d3205235d5cfab9c24"}
Nov 03 16:10:05.831485 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:05.831459    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8k9vh" event={"ID":"38701b60-1908-4568-81ed-0c5a91716678","Type":"ContainerStarted","Data":"d7131ffb75c656b472b63663686baf4bc4e29c6b45b7bd1038639fe5b3766b23"}
Nov 03 16:10:05.957393 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:05.957332    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-g2fnp" podStartSLOduration=7.716693645 podStartE2EDuration="27.957319907s" totalImagesPullingTime="20.240626262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:09:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:10:05.95670878 +0000 UTC m=+28.789349455" watchObservedRunningTime="2026-11-03 16:10:05.957319907 +0000 UTC m=+28.789960578"
Nov 03 16:10:06.077686 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:06.077594    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8k9vh" podStartSLOduration=2.675284248 podStartE2EDuration="28.077576021s" totalImagesPullingTime="25.402291773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:10:06.076924096 +0000 UTC m=+28.909564768" watchObservedRunningTime="2026-11-03 16:10:06.077576021 +0000 UTC m=+28.910216694"
Nov 03 16:10:06.711925 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:06.711890    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:06.712091 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:06.712027    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:07.712382 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:07.712350    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:07.712798 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:07.712478    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:07.712798 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:07.712537    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:07.712798 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:07.712634    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:08.712166 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:08.712141    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:08.712322 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:08.712234    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:09.712510 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:09.712474    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:09.712510 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:09.712504    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:09.712972 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:09.712588    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:09.712972 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:09.712659    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:10.711396 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:10.711362    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:10.711557 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:10.711466    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:11.546183 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:11.546146    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:11.546674 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:11.546205    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:11.546674 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:11.546232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:11.546674 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:11.546321    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 16:10:11.546674 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:11.546346    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 16:10:11.546674 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:11.546345    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 16:10:11.546674 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:11.546352    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:10:11.546674 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:11.546357    2582 projected.go:196] Error preparing data for projected volume kube-api-access-zsxr9 for pod openshift-network-diagnostics/network-check-target-lqvng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:10:11.546674 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:11.546408    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret podName:c6f6992c-48b9-4bb5-9b85-a37552b64c8b nodeName:}" failed. No retries permitted until 2026-11-03 16:10:43.546391198 +0000 UTC m=+66.379031862 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret") pod "global-pull-secret-syncer-tbpr9" (UID: "c6f6992c-48b9-4bb5-9b85-a37552b64c8b") : object "kube-system"/"original-pull-secret" not registered
Nov 03 16:10:11.546674 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:11.546427    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9 podName:a67334f9-d13e-452c-b266-e0e996dd7db3 nodeName:}" failed. No retries permitted until 2026-11-03 16:10:43.546415549 +0000 UTC m=+66.379056239 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zsxr9" (UniqueName: "kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9") pod "network-check-target-lqvng" (UID: "a67334f9-d13e-452c-b266-e0e996dd7db3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:10:11.546674 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:11.546441    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs podName:cffd0b63-ba73-433f-886f-fcae960b41a1 nodeName:}" failed. No retries permitted until 2026-11-03 16:10:43.546434557 +0000 UTC m=+66.379075211 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs") pod "network-metrics-daemon-k5rqd" (UID: "cffd0b63-ba73-433f-886f-fcae960b41a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:10:11.712430 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:11.712398    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:11.712606 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:11.712398    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:11.712606 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:11.712516    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:11.712606 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:11.712596    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:12.712216 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:12.712184    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:12.712610 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:12.712281    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:13.712260 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:13.712230    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:13.712621 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:13.712234    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:13.712621 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:13.712333    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:13.712621 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:13.712428    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:14.711869 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:14.711840    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:14.712037 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:14.711930    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:15.712222 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:15.712184    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:15.712689 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:15.712339    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:15.712689 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:15.712376    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:15.712689 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:15.712455    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:16.712294 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:16.712248    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:16.712683 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:16.712360    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:17.712107 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:17.712075    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:17.712291 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:17.712182    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:17.712291 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:17.712257    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:17.712623 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:17.712378    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:18.712361 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:18.712329    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:18.712751 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:18.712427    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:19.712170 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:19.712139    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:19.712328 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:19.712284    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:19.712371 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:19.712324    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:19.712436 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:19.712420    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:20.711740 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:20.711704    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:20.711904 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:20.711818    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:21.711771 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:21.711735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:21.712214 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:21.711876    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:21.712214 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:21.711937    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:21.712214 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:21.712067    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:22.711590 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:22.711552    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:22.711800 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:22.711694    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:23.712382 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:23.712348    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:23.712864 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:23.712396    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:23.712864 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:23.712504    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:23.712864 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:23.712653    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:24.712219 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:24.712041    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:24.712386 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:24.712293    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:24.871487 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:24.871455    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l" event={"ID":"1897a95d-8237-4a69-9ae8-c229490ba88c","Type":"ContainerStarted","Data":"bd669bc5c74c0d4747ef41f098ee52cdb64931e78b166294cf4a44697b902d18"}
Nov 03 16:10:24.872011 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:24.871519    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l" event={"ID":"1897a95d-8237-4a69-9ae8-c229490ba88c","Type":"ContainerStarted","Data":"c3eb7afd8fe10b53bfaa8211fb230f73bc2ba3b768117180e58e5bd8f1e6b4cd"}
Nov 03 16:10:24.872011 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:24.871549    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l" event={"ID":"1897a95d-8237-4a69-9ae8-c229490ba88c","Type":"ContainerStarted","Data":"84c74c94a878e27b57ab98a7775734c1280fe425a8507014f15252607102d592"}
Nov 03 16:10:24.872011 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:24.871577    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l" event={"ID":"1897a95d-8237-4a69-9ae8-c229490ba88c","Type":"ContainerStarted","Data":"a73bd1a9223a2c30e06c57ce077751ce41b3d07751ced16bc6847205db64c2b8"}
Nov 03 16:10:24.872011 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:24.871603    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l" event={"ID":"1897a95d-8237-4a69-9ae8-c229490ba88c","Type":"ContainerStarted","Data":"fcf8c60e625c58bfc3130ee966f4d8d07eeb992f36fb1dbecaf20bd037e3980e"}
Nov 03 16:10:24.872011 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:24.871629    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l" event={"ID":"1897a95d-8237-4a69-9ae8-c229490ba88c","Type":"ContainerStarted","Data":"05813596f01ccccf6a86fbb2ea6cec82c9fed396881c8636c7161d884b370a32"}
Nov 03 16:10:25.711893 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:25.711863    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:25.711893 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:25.711902    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:25.712105 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:25.712010    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:25.712142 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:25.712122    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:26.712051 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:26.712024    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:26.712475 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:26.712123    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:26.877987 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:26.877904    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l" event={"ID":"1897a95d-8237-4a69-9ae8-c229490ba88c","Type":"ContainerStarted","Data":"f385b1d5e69a49bd38b7e868b4d0f8f6bdb0909d922799a7a69e9ca0fb14ffa5"}
Nov 03 16:10:27.712686 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:27.712652    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:27.713118 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:27.712828    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:27.713118 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:27.712879    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:27.713118 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:27.712968    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:28.712366 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:28.712176    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:28.712366 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:28.712278    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:28.886403 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:28.886223    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l" event={"ID":"1897a95d-8237-4a69-9ae8-c229490ba88c","Type":"ContainerStarted","Data":"06301011028c3fe2dbec8a0b24a6d4c8c936766c8e55b25e5fce5882b172356d"}
Nov 03 16:10:28.886954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:28.886557    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:10:28.886954 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:28.886628    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:10:28.903438 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:28.903417    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:10:29.169404 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:29.169340    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l" podStartSLOduration=6.98930214 podStartE2EDuration="51.169328122s" totalImagesPullingTime="44.180025982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:10:29.119350891 +0000 UTC m=+51.951991566" watchObservedRunningTime="2026-11-03 16:10:29.169328122 +0000 UTC m=+52.001968795"
Nov 03 16:10:29.712120 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:29.712085    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:29.712327 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:29.712234    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:29.712327 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:29.712276    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:29.712441 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:29.712371    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:29.888746 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:29.888697    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:10:29.907127 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:29.907102    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:10:30.712198 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:30.712171    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:30.712415 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:30.712270    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:31.371330 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:31.371299    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tbpr9"]
Nov 03 16:10:31.371765 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:31.371410    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:31.371765 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:31.371498    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:31.374131 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:31.374107    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lqvng"]
Nov 03 16:10:31.374236 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:31.374186    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:31.374282 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:31.374258    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:31.374579 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:31.374562    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k5rqd"]
Nov 03 16:10:31.374650 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:31.374639    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:31.374749 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:31.374713    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:32.711796 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:32.711755    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:32.712315 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:32.711818    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:32.712315 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:32.711906    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:32.712315 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:32.711919    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:32.712315 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:32.711976    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:32.712315 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:32.712046    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:33.106912 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:33.106888    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j8mcw_0599707f-1cdb-4e9f-99da-72703b028198/dns-node-resolver/0.log"
Nov 03 16:10:33.942473 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:33.942441    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gfrfv_d80b9f00-54a1-4809-9856-efb0ee40bd24/node-ca/0.log"
Nov 03 16:10:34.712273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:34.712243    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:34.712273 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:34.712277    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:34.712477 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:34.712354    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:34.712477 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:34.712362    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:34.712477 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:34.712445    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:34.712564 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:34.712532    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:34.899801 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:34.899777    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:10:34.899936 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:34.899816    2582 generic.go:356] "Generic (PLEG): container finished" podID="1c560131-5ae7-42ef-975b-23af6909fb89" containerID="b4f769c479f8a101ebab235dce7f85580ec680ba9c88955be5d8b1414879ab17" exitCode=1
Nov 03 16:10:34.899936 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:34.899876    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qgfrq" event={"ID":"1c560131-5ae7-42ef-975b-23af6909fb89","Type":"ContainerDied","Data":"b4f769c479f8a101ebab235dce7f85580ec680ba9c88955be5d8b1414879ab17"}
Nov 03 16:10:34.900300 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:34.900285    2582 scope.go:122] "RemoveContainer" containerID="b4f769c479f8a101ebab235dce7f85580ec680ba9c88955be5d8b1414879ab17"
Nov 03 16:10:35.903992 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:35.903833    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:10:35.904362 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:35.904027    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qgfrq" event={"ID":"1c560131-5ae7-42ef-975b-23af6909fb89","Type":"ContainerStarted","Data":"7bc5204a0d55d1e0f9d74eabd780ee0ede70463785900739dc50828636de24e0"}
Nov 03 16:10:36.711643 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:36.711614    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:36.711643 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:36.711634    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:36.711980 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:36.711666    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:36.711980 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:36.711750    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:36.711980 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:36.711808    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:36.711980 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:36.711915    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:38.712248 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:38.712209    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:38.712950 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:38.712284    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:38.712950 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:38.712387    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:38.712950 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:38.712402    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:38.712950 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:38.712465    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:38.712950 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:38.712511    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:40.712345 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:40.712305    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:40.712345 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:40.712354    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:40.712812 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:40.712421    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:40.712812 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:40.712440    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:40.712812 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:40.712495    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:40.712812 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:40.712561    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:42.712347 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:42.712299    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:42.712347 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:42.712330    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:42.712796 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:42.712415    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:42.712796 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:42.712430    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:42.712796 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:42.712477    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:42.712796 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:42.712558    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:43.561538 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:43.561500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:43.561538 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:43.561543    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:43.561769 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:43.561596    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:43.561769 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:43.561664    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:10:43.561769 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:43.561702    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 16:10:43.561769 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:43.561716    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 16:10:43.561769 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:43.561718    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 16:10:43.561769 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:43.561751    2582 projected.go:196] Error preparing data for projected volume kube-api-access-zsxr9 for pod openshift-network-diagnostics/network-check-target-lqvng: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:10:43.561944 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:43.561757    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs podName:cffd0b63-ba73-433f-886f-fcae960b41a1 nodeName:}" failed. No retries permitted until 2026-11-03 16:11:47.5617134 +0000 UTC m=+130.394354068 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs") pod "network-metrics-daemon-k5rqd" (UID: "cffd0b63-ba73-433f-886f-fcae960b41a1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 16:10:43.561944 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:43.561800    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret podName:c6f6992c-48b9-4bb5-9b85-a37552b64c8b nodeName:}" failed. No retries permitted until 2026-11-03 16:11:47.561790542 +0000 UTC m=+130.394431201 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret") pod "global-pull-secret-syncer-tbpr9" (UID: "c6f6992c-48b9-4bb5-9b85-a37552b64c8b") : object "kube-system"/"original-pull-secret" not registered
Nov 03 16:10:43.561944 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:43.561813    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9 podName:a67334f9-d13e-452c-b266-e0e996dd7db3 nodeName:}" failed. No retries permitted until 2026-11-03 16:11:47.561805216 +0000 UTC m=+130.394445867 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zsxr9" (UniqueName: "kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9") pod "network-check-target-lqvng" (UID: "a67334f9-d13e-452c-b266-e0e996dd7db3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 16:10:44.712239 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:44.712211    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:44.712641 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:44.712216    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:44.712641 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:44.712305    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:44.712641 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:44.712217    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:44.712641 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:44.712383    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:44.712641 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:44.712459    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:46.712070 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:46.712021    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:46.712070 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:46.712044    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:46.712485 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:46.712021    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:46.712485 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:46.712133    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:46.712485 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:46.712194    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:46.712485 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:46.712276    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:48.712268 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:48.712237    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:48.712268 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:48.712250    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:48.712862 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:48.712339    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:48.712862 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:48.712345    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:48.712862 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:48.712452    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:48.712862 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:48.712526    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:50.711635 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:50.711604    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:50.712039 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:50.711604    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:50.712039 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:50.711708    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:50.712039 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:50.711815    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:50.712039 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:50.711610    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:50.712039 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:50.711906    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:52.711781 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:52.711746    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:52.712192 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:52.711858    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:52.712192 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:52.711887    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:52.712192 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:52.711962    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:52.712192 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:52.712006    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:52.712192 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:52.712063    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:54.712302 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:54.712271    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:54.712760 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:54.712366    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lqvng" podUID="a67334f9-d13e-452c-b266-e0e996dd7db3"
Nov 03 16:10:54.712760 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:54.712372    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:54.712760 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:54.712275    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:54.712760 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:54.712469    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbpr9" podUID="c6f6992c-48b9-4bb5-9b85-a37552b64c8b"
Nov 03 16:10:54.712760 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:10:54.712554    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k5rqd" podUID="cffd0b63-ba73-433f-886f-fcae960b41a1"
Nov 03 16:10:54.951317 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:54.951288    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeReady"
Nov 03 16:10:54.951489 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:54.951399    2582 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 03 16:10:55.111903 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.111865    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ffzs5"]
Nov 03 16:10:55.131795 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.131767    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kbql5"]
Nov 03 16:10:55.131936 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.131888    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.149842 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.149817    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7ltc8"]
Nov 03 16:10:55.149952 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.149861    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:10:55.164034 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.164008    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ffzs5"]
Nov 03 16:10:55.164162 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.164061    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kbql5"]
Nov 03 16:10:55.164162 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.164130    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.189312 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.189284    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 16:10:55.189476 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.189463    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7zz2z\""
Nov 03 16:10:55.189526 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.189513    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 16:10:55.193401 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.193377    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7ltc8"]
Nov 03 16:10:55.235156 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235077    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzk2k\" (UniqueName: \"kubernetes.io/projected/54d61260-b3aa-46f1-8cdd-d15102a208ce-kube-api-access-wzk2k\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.235156 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235112    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-cert\") pod \"ingress-canary-kbql5\" (UID: \"d3a76124-dbf8-4fb3-bc58-52a21819b9d9\") " pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:10:55.235156 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235130    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl42j\" (UniqueName: \"kubernetes.io/projected/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-api-access-jl42j\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.235357 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235210    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ffeba40d-6416-4f6f-b645-d6e3338f7adb-crio-socket\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.235357 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235244    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffeba40d-6416-4f6f-b645-d6e3338f7adb-data-volume\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.235357 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235270    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d61260-b3aa-46f1-8cdd-d15102a208ce-tmp-dir\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.235357 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235331    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ffeba40d-6416-4f6f-b645-d6e3338f7adb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.235473 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235363    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.235473 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54d61260-b3aa-46f1-8cdd-d15102a208ce-config-volume\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.235473 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235393    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54d61260-b3aa-46f1-8cdd-d15102a208ce-metrics-tls\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.235473 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.235439    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z56b7\" (UniqueName: \"kubernetes.io/projected/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-kube-api-access-z56b7\") pod \"ingress-canary-kbql5\" (UID: \"d3a76124-dbf8-4fb3-bc58-52a21819b9d9\") " pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:10:55.290329 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.290296    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 16:10:55.290482 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.290356    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 16:10:55.290557 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.290480    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 16:10:55.290557 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.290516    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 16:10:55.290625 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.290582    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 16:10:55.290625 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.290586    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4pfhl\""
Nov 03 16:10:55.290625 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.290616    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 16:10:55.290752 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.290637    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-gswn6\""
Nov 03 16:10:55.290752 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.290646    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 16:10:55.336719 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336691    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ffeba40d-6416-4f6f-b645-d6e3338f7adb-crio-socket\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.336834 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffeba40d-6416-4f6f-b645-d6e3338f7adb-data-volume\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.336834 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336772    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d61260-b3aa-46f1-8cdd-d15102a208ce-tmp-dir\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.336834 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ffeba40d-6416-4f6f-b645-d6e3338f7adb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.336834 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336822    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ffeba40d-6416-4f6f-b645-d6e3338f7adb-crio-socket\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.336987 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336842    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.336987 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54d61260-b3aa-46f1-8cdd-d15102a208ce-config-volume\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.336987 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336885    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54d61260-b3aa-46f1-8cdd-d15102a208ce-metrics-tls\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.336987 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z56b7\" (UniqueName: \"kubernetes.io/projected/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-kube-api-access-z56b7\") pod \"ingress-canary-kbql5\" (UID: \"d3a76124-dbf8-4fb3-bc58-52a21819b9d9\") " pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:10:55.336987 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336974    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzk2k\" (UniqueName: \"kubernetes.io/projected/54d61260-b3aa-46f1-8cdd-d15102a208ce-kube-api-access-wzk2k\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.337220 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.336996    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-cert\") pod \"ingress-canary-kbql5\" (UID: \"d3a76124-dbf8-4fb3-bc58-52a21819b9d9\") " pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:10:55.337220 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.337019    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl42j\" (UniqueName: \"kubernetes.io/projected/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-api-access-jl42j\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.337220 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.337061    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffeba40d-6416-4f6f-b645-d6e3338f7adb-data-volume\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.337220 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.337175    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d61260-b3aa-46f1-8cdd-d15102a208ce-tmp-dir\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.337680 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.337659    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.337864 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.337848    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54d61260-b3aa-46f1-8cdd-d15102a208ce-config-volume\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.340780 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.340756    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ffeba40d-6416-4f6f-b645-d6e3338f7adb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.340933 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.340917    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54d61260-b3aa-46f1-8cdd-d15102a208ce-metrics-tls\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.340992 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.340954    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-cert\") pod \"ingress-canary-kbql5\" (UID: \"d3a76124-dbf8-4fb3-bc58-52a21819b9d9\") " pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:10:55.397784 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.397754    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z56b7\" (UniqueName: \"kubernetes.io/projected/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-kube-api-access-z56b7\") pod \"ingress-canary-kbql5\" (UID: \"d3a76124-dbf8-4fb3-bc58-52a21819b9d9\") " pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:10:55.398009 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.397992    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl42j\" (UniqueName: \"kubernetes.io/projected/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-api-access-jl42j\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:55.398290 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.398266    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzk2k\" (UniqueName: \"kubernetes.io/projected/54d61260-b3aa-46f1-8cdd-d15102a208ce-kube-api-access-wzk2k\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.464402 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.464364    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:55.482510 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.482484    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:10:55.533376 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:55.533351    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:10:56.018921 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.018896    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kbql5"]
Nov 03 16:10:56.019572 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:10:56.019547    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a76124_dbf8_4fb3_bc58_52a21819b9d9.slice/crio-edbb68bb5cbcfaf8e36384356bc1f636bfc4922eaebe1e7d75ae041fd7ca91db WatchSource:0}: Error finding container edbb68bb5cbcfaf8e36384356bc1f636bfc4922eaebe1e7d75ae041fd7ca91db: Status 404 returned error can't find the container with id edbb68bb5cbcfaf8e36384356bc1f636bfc4922eaebe1e7d75ae041fd7ca91db
Nov 03 16:10:56.019823 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:10:56.019804    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d61260_b3aa_46f1_8cdd_d15102a208ce.slice/crio-839386ca47a23cf94d69213f6f6395de4fa713f3bfb6dd4b5192343ab1c0c969 WatchSource:0}: Error finding container 839386ca47a23cf94d69213f6f6395de4fa713f3bfb6dd4b5192343ab1c0c969: Status 404 returned error can't find the container with id 839386ca47a23cf94d69213f6f6395de4fa713f3bfb6dd4b5192343ab1c0c969
Nov 03 16:10:56.019932 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.019914    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ffzs5"]
Nov 03 16:10:56.020518 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.020498    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7ltc8"]
Nov 03 16:10:56.061059 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:10:56.061034    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffeba40d_6416_4f6f_b645_d6e3338f7adb.slice/crio-af1e5abda95474af2c5ae13d8bfbe3d225bd8c2911b774b657c401c97b3f0534 WatchSource:0}: Error finding container af1e5abda95474af2c5ae13d8bfbe3d225bd8c2911b774b657c401c97b3f0534: Status 404 returned error can't find the container with id af1e5abda95474af2c5ae13d8bfbe3d225bd8c2911b774b657c401c97b3f0534
Nov 03 16:10:56.711978 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.711932    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:10:56.711978 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.711959    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:10:56.712219 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.711964    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:10:56.767848 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.767828    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 16:10:56.767848 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.767831    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 16:10:56.768920 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.768895    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 16:10:56.819026 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.818993    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zsc2l\""
Nov 03 16:10:56.819026 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.819019    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rqr4v\""
Nov 03 16:10:56.819167 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.818995    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-j7zlt\""
Nov 03 16:10:56.819167 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.819053    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 16:10:56.947487 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.947449    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7ltc8" event={"ID":"ffeba40d-6416-4f6f-b645-d6e3338f7adb","Type":"ContainerStarted","Data":"288e762e1f3f5ddf635346c6eb4b3f4091698b27e13c0ad9e90d94326b241a6d"}
Nov 03 16:10:56.947644 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.947513    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7ltc8" event={"ID":"ffeba40d-6416-4f6f-b645-d6e3338f7adb","Type":"ContainerStarted","Data":"af1e5abda95474af2c5ae13d8bfbe3d225bd8c2911b774b657c401c97b3f0534"}
Nov 03 16:10:56.948562 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.948537    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kbql5" event={"ID":"d3a76124-dbf8-4fb3-bc58-52a21819b9d9","Type":"ContainerStarted","Data":"edbb68bb5cbcfaf8e36384356bc1f636bfc4922eaebe1e7d75ae041fd7ca91db"}
Nov 03 16:10:56.949564 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:56.949538    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ffzs5" event={"ID":"54d61260-b3aa-46f1-8cdd-d15102a208ce","Type":"ContainerStarted","Data":"839386ca47a23cf94d69213f6f6395de4fa713f3bfb6dd4b5192343ab1c0c969"}
Nov 03 16:10:57.651485 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.651445    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dd847f68b-wq7jc"]
Nov 03 16:10:57.661332 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.661307    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.717147 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.717116    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 03 16:10:57.756070 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.756045    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-config\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.756204 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.756085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-serving-cert\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.756204 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.756115    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbshk\" (UniqueName: \"kubernetes.io/projected/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-kube-api-access-mbshk\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.756389 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.756372    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-oauth-config\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.756458 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.756413    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-service-ca\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.756517 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.756498    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-trusted-ca-bundle\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.756635 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.756620    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-tmp\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.756713 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.756655    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-oauth-serving-cert\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.766951 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.766922    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 03 16:10:57.767058 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.766998    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 03 16:10:57.767274 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.767252    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 03 16:10:57.767361 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.767311    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 03 16:10:57.768142 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.768118    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 03 16:10:57.818062 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.818030    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jkpm9\""
Nov 03 16:10:57.818309 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.818287    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 03 16:10:57.858011 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.857985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-oauth-config\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.858178 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.858021    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-service-ca\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.858178 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.858048    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-trusted-ca-bundle\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.858178 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.858074    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-tmp\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.858178 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.858101    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-oauth-serving-cert\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.858178 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.858141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-config\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.858432 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.858171    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-serving-cert\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.858432 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.858207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbshk\" (UniqueName: \"kubernetes.io/projected/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-kube-api-access-mbshk\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.858800 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.858776    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-service-ca\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.858800 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.858794    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-tmp\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.859046 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.859026    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-oauth-serving-cert\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.859095 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.859058    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-config\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.861168 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.861148    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-oauth-config\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.861251 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.861182    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-serving-cert\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.870764 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.870714    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 03 16:10:57.871557 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.871537    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dd847f68b-wq7jc"]
Nov 03 16:10:57.879542 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.879516    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-trusted-ca-bundle\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.918597 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.918529    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbshk\" (UniqueName: \"kubernetes.io/projected/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-kube-api-access-mbshk\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:57.953993 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.953950    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7ltc8" event={"ID":"ffeba40d-6416-4f6f-b645-d6e3338f7adb","Type":"ContainerStarted","Data":"9cb87791fdb85016ec5908ae7c4ec9c254b2181d817d879885003fbc5f05258a"}
Nov 03 16:10:57.984393 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:57.984362    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:10:58.215343 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.215271    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"]
Nov 03 16:10:58.236138 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.236108    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.275525 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.275492    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"]
Nov 03 16:10:58.292631 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.292387    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 16:10:58.292631 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.292387    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 16:10:58.292631 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.292387    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 03 16:10:58.292631 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.292573    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 03 16:10:58.292631 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.292602    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 16:10:58.293103 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.292689    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-x8k5l\""
Nov 03 16:10:58.362739 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.362706    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.362914 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.362758    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9693f80-8f98-41c3-a72e-089d647d47a2-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.362914 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.362864    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.362992 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.362967    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwx5m\" (UniqueName: \"kubernetes.io/projected/c9693f80-8f98-41c3-a72e-089d647d47a2-kube-api-access-zwx5m\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.443641 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.443614    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dd847f68b-wq7jc"]
Nov 03 16:10:58.449281 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:10:58.449251    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbafe6e21_68bb_4c42_84bf_96e38b7c1e55.slice/crio-eb0e217e600d3bdca1e147cbb5263225a859fa5159e34f9499c07b7567c1396c WatchSource:0}: Error finding container eb0e217e600d3bdca1e147cbb5263225a859fa5159e34f9499c07b7567c1396c: Status 404 returned error can't find the container with id eb0e217e600d3bdca1e147cbb5263225a859fa5159e34f9499c07b7567c1396c
Nov 03 16:10:58.463329 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.463282    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.463329 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.463319    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9693f80-8f98-41c3-a72e-089d647d47a2-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.463442 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.463371    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.463917 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.463887    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwx5m\" (UniqueName: \"kubernetes.io/projected/c9693f80-8f98-41c3-a72e-089d647d47a2-kube-api-access-zwx5m\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.464520 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.464498    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9693f80-8f98-41c3-a72e-089d647d47a2-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.466222 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.466159    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.466388 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.466362    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.524442 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.524413    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwx5m\" (UniqueName: \"kubernetes.io/projected/c9693f80-8f98-41c3-a72e-089d647d47a2-kube-api-access-zwx5m\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.571820 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.571784    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:10:58.958848 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.958813    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dd847f68b-wq7jc" event={"ID":"bafe6e21-68bb-4c42-84bf-96e38b7c1e55","Type":"ContainerStarted","Data":"eb0e217e600d3bdca1e147cbb5263225a859fa5159e34f9499c07b7567c1396c"}
Nov 03 16:10:58.961776 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.961659    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ffzs5" event={"ID":"54d61260-b3aa-46f1-8cdd-d15102a208ce","Type":"ContainerStarted","Data":"dce22af84fc8155ffdf3473fe96facbd608d8f9850416caaf3528f387473237b"}
Nov 03 16:10:58.961776 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.961718    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ffzs5" event={"ID":"54d61260-b3aa-46f1-8cdd-d15102a208ce","Type":"ContainerStarted","Data":"3518bd37f480374e137641916200496f1819db62bf25963533c0bf3ae8b137a2"}
Nov 03 16:10:58.961995 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:58.961977    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:10:59.065295 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:10:59.065268    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9693f80_8f98_41c3_a72e_089d647d47a2.slice/crio-7f6d93c7656545212da5ac11e453f70ab88ee42e872c14e6c466cd8dbe78ad2c WatchSource:0}: Error finding container 7f6d93c7656545212da5ac11e453f70ab88ee42e872c14e6c466cd8dbe78ad2c: Status 404 returned error can't find the container with id 7f6d93c7656545212da5ac11e453f70ab88ee42e872c14e6c466cd8dbe78ad2c
Nov 03 16:10:59.065699 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:59.065672    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"]
Nov 03 16:10:59.085190 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:59.084969    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-ffzs5" podStartSLOduration=1.6468137440000001 podStartE2EDuration="4.084950023s" totalImagesPullingTime="2.438136279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:10:59.084052704 +0000 UTC m=+81.916693379" watchObservedRunningTime="2026-11-03 16:10:59.084950023 +0000 UTC m=+81.917590699"
Nov 03 16:10:59.967365 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:59.967272    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7ltc8" event={"ID":"ffeba40d-6416-4f6f-b645-d6e3338f7adb","Type":"ContainerStarted","Data":"7a05601fb21e78837b0fdef220187769e9614259daf612663b135c83324b0b85"}
Nov 03 16:10:59.968763 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:10:59.968718    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5" event={"ID":"c9693f80-8f98-41c3-a72e-089d647d47a2","Type":"ContainerStarted","Data":"7f6d93c7656545212da5ac11e453f70ab88ee42e872c14e6c466cd8dbe78ad2c"}
Nov 03 16:11:00.088387 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:00.088338    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7ltc8" podStartSLOduration=1.649584139 podStartE2EDuration="5.088325026s" totalImagesPullingTime="3.438740887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 16:10:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:11:00.0881037 +0000 UTC m=+82.920744371" watchObservedRunningTime="2026-11-03 16:11:00.088325026 +0000 UTC m=+82.920965698"
Nov 03 16:11:00.972588 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:00.972547    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kbql5" event={"ID":"d3a76124-dbf8-4fb3-bc58-52a21819b9d9","Type":"ContainerStarted","Data":"935aba99cde154e5398990dd08f8fc78af5852e21080a306853ce7821b0f44bf"}
Nov 03 16:11:01.093531 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:01.093481    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kbql5" podStartSLOduration=1.757177114 podStartE2EDuration="6.093464443s" totalImagesPullingTime="4.336287329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:11:01.092303523 +0000 UTC m=+83.924944198" watchObservedRunningTime="2026-11-03 16:11:01.093464443 +0000 UTC m=+83.926105116"
Nov 03 16:11:01.906627 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:01.906599    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:02.984426 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:02.984322    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dd847f68b-wq7jc" event={"ID":"bafe6e21-68bb-4c42-84bf-96e38b7c1e55","Type":"ContainerStarted","Data":"289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d"}
Nov 03 16:11:02.986251 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:02.986228    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5" event={"ID":"c9693f80-8f98-41c3-a72e-089d647d47a2","Type":"ContainerStarted","Data":"ffdf55f32790dad0a1f2714e819695f80ff0b0a932862bbcbf71b6d5f66e89a9"}
Nov 03 16:11:02.986377 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:02.986279    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5" event={"ID":"c9693f80-8f98-41c3-a72e-089d647d47a2","Type":"ContainerStarted","Data":"21fbf5196314055831e8607515febcb9f07537884e0807c8ba4e484a44ee0fd4"}
Nov 03 16:11:03.107660 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:03.107613    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7dd847f68b-wq7jc" podStartSLOduration=1.955585146 podStartE2EDuration="6.107601898s" totalImagesPullingTime="4.152016752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:10:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:11:03.107344605 +0000 UTC m=+85.939985277" watchObservedRunningTime="2026-11-03 16:11:03.107601898 +0000 UTC m=+85.940242569"
Nov 03 16:11:03.228749 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:03.228685    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5" podStartSLOduration=2.090306202 podStartE2EDuration="5.228670861s" totalImagesPullingTime="3.138364659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:11:03.228110086 +0000 UTC m=+86.060750762" watchObservedRunningTime="2026-11-03 16:11:03.228670861 +0000 UTC m=+86.061311533"
Nov 03 16:11:07.072117 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.072084    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rbcxk"]
Nov 03 16:11:07.074487 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.074471    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.134098 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.134072    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-root\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.134098 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.134102    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-textfile\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.134268 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.134131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-sys\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.134268 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.134239    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.134328 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.134306    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-metrics-client-ca\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.134358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.134338    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-tls\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.134358 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.134355    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-wtmp\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.134413 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.134385    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.134444 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.134413    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh6ld\" (UniqueName: \"kubernetes.io/projected/8ad82d3f-4298-4eb1-979b-91f94c007cef-kube-api-access-hh6ld\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.196401 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.196369    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 16:11:07.196756 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.196720    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m7glm\""
Nov 03 16:11:07.196824 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.196809    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 16:11:07.197166 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.197150    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 16:11:07.235611 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.235583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-metrics-client-ca\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.235765 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.235621    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-tls\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.235765 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.235640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-wtmp\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.235765 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.235668    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.235765 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.235685    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh6ld\" (UniqueName: \"kubernetes.io/projected/8ad82d3f-4298-4eb1-979b-91f94c007cef-kube-api-access-hh6ld\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.235960 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.235770    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-root\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.235960 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.235796    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-textfile\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.235960 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.235830    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-sys\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.235960 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.235846    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-wtmp\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.235960 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.235854    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-root\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.236388 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.236359    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.236638 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.236500    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-textfile\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.236783 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.236764    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-sys\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.236896 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.236868    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-metrics-client-ca\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.236990 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.236763    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.239006 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.238981    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-tls\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.242739 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.242701    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.300030 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.300002    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh6ld\" (UniqueName: \"kubernetes.io/projected/8ad82d3f-4298-4eb1-979b-91f94c007cef-kube-api-access-hh6ld\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.387167 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.387132    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:07.396883 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:11:07.396848    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad82d3f_4298_4eb1_979b_91f94c007cef.slice/crio-f5a5565102010c4d5acf9f84f18c149ae4f0c16d9461cddbc8ff8e6bf14f7795 WatchSource:0}: Error finding container f5a5565102010c4d5acf9f84f18c149ae4f0c16d9461cddbc8ff8e6bf14f7795: Status 404 returned error can't find the container with id f5a5565102010c4d5acf9f84f18c149ae4f0c16d9461cddbc8ff8e6bf14f7795
Nov 03 16:11:07.984908 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.984868    2582 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:07.985078 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.984945    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:07.990141 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:07.990118    2582 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:08.002241 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:08.002212    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbcxk" event={"ID":"8ad82d3f-4298-4eb1-979b-91f94c007cef","Type":"ContainerStarted","Data":"f5a5565102010c4d5acf9f84f18c149ae4f0c16d9461cddbc8ff8e6bf14f7795"}
Nov 03 16:11:08.006055 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:08.006034    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:08.971150 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:08.971109    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:11:09.345598 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.345563    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 16:11:09.348931 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.348910    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.405021 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.404980    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 03 16:11:09.405021 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.405014    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 03 16:11:09.405230 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.405002    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 03 16:11:09.405230 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.405019    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 03 16:11:09.405230 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.404980    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cbsvm\""
Nov 03 16:11:09.405230 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.405055    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 03 16:11:09.405230 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.404980    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 03 16:11:09.405230 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.405018    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 03 16:11:09.405465 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.405442    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 03 16:11:09.454633 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.454575    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.454807 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.454663    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.454807 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.454701    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddv4f\" (UniqueName: \"kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-kube-api-access-ddv4f\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.454931 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.454804    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.454931 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.454835    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.455051 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.454927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.455051 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.455000    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-web-config\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.455152 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.455064    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.455152 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.455121    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.455152 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.455144    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.455253 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.455174    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.455301 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.455276    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-config-out\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.455347 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.455325    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-config-volume\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.456185 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.456149    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 16:11:09.506231 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.506193    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 03 16:11:09.555748 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.555709    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-web-config\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.555854 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.555767    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.555854 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.555825    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.555854 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.555849    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.556009 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.555874    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.556009 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.555918    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-config-out\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.556009 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.555950    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-config-volume\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.556009 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.555997    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.556199 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.556025    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.556199 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.556052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddv4f\" (UniqueName: \"kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-kube-api-access-ddv4f\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.556199 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.556096    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.556199 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.556121    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.556199 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.556148    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.556441 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.556236    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.557362 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.556569    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.557362 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.557124    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.558846 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.558819    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-config-out\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.559158 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.559133    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-web-config\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.559436 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.559413    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.559636 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.559613    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-config-volume\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.559712 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.559635    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.559814 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.559802    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.560141 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.560121    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.560421 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.560404    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.560657 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.560639    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.616408 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.616377    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddv4f\" (UniqueName: \"kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-kube-api-access-ddv4f\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:09.675513 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:09.675485    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:10.009974 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:10.009889    2582 generic.go:356] "Generic (PLEG): container finished" podID="8ad82d3f-4298-4eb1-979b-91f94c007cef" containerID="5caaeae2e6d2f0075e2e8f4da6fd3215b50ea729b789746bfa1469c3e3bb514d" exitCode=0
Nov 03 16:11:10.009974 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:10.009947    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbcxk" event={"ID":"8ad82d3f-4298-4eb1-979b-91f94c007cef","Type":"ContainerDied","Data":"5caaeae2e6d2f0075e2e8f4da6fd3215b50ea729b789746bfa1469c3e3bb514d"}
Nov 03 16:11:10.099524 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:10.099505    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 16:11:10.104782 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:11:10.104760    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648f1fa2_0c16_4abc_964b_200caa87def3.slice/crio-3019d8ce3631c97e275f3646298176c2f6355d9291eb32d4ada3fd0e87496b55 WatchSource:0}: Error finding container 3019d8ce3631c97e275f3646298176c2f6355d9291eb32d4ada3fd0e87496b55: Status 404 returned error can't find the container with id 3019d8ce3631c97e275f3646298176c2f6355d9291eb32d4ada3fd0e87496b55
Nov 03 16:11:11.016906 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:11.016869    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"648f1fa2-0c16-4abc-964b-200caa87def3","Type":"ContainerStarted","Data":"3019d8ce3631c97e275f3646298176c2f6355d9291eb32d4ada3fd0e87496b55"}
Nov 03 16:11:11.018979 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:11.018954    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbcxk" event={"ID":"8ad82d3f-4298-4eb1-979b-91f94c007cef","Type":"ContainerStarted","Data":"c83b013d3be8316df0585583e485605d9b539861692e4c53d81f983a9dbfd087"}
Nov 03 16:11:11.019105 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:11.019000    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbcxk" event={"ID":"8ad82d3f-4298-4eb1-979b-91f94c007cef","Type":"ContainerStarted","Data":"08b80bdcf0b2a733c966fee9336843242f76f301566d3d338b1b620dc572411d"}
Nov 03 16:11:11.141371 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:11.141315    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rbcxk" podStartSLOduration=1.9256834170000001 podStartE2EDuration="4.14130056s" totalImagesPullingTime="2.138753926s" totalInitContainerRuntime="76.863217ms" isStatefulPod=true podCreationTimestamp="2026-11-03 16:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:11:11.140410898 +0000 UTC m=+93.973051569" watchObservedRunningTime="2026-11-03 16:11:11.14130056 +0000 UTC m=+93.973941231"
Nov 03 16:11:12.028068 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:12.027974    2582 generic.go:356] "Generic (PLEG): container finished" podID="648f1fa2-0c16-4abc-964b-200caa87def3" containerID="cda4f17ff7aa7d71b14326ec61f99c62b17d2aef5bca76098587796bf653cbd8" exitCode=0
Nov 03 16:11:12.028421 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:12.028058    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"648f1fa2-0c16-4abc-964b-200caa87def3","Type":"ContainerDied","Data":"cda4f17ff7aa7d71b14326ec61f99c62b17d2aef5bca76098587796bf653cbd8"}
Nov 03 16:11:15.040841 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:15.040758    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"648f1fa2-0c16-4abc-964b-200caa87def3","Type":"ContainerStarted","Data":"8b7b382f8ba8a98b309e3b540d2463d9f62b62599ca9bb3c76ed9ed88aab32e7"}
Nov 03 16:11:15.040841 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:15.040825    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"648f1fa2-0c16-4abc-964b-200caa87def3","Type":"ContainerStarted","Data":"c31481b4b6079b77f05997c701d3ace909dfe003f926773a1028d3d28da1505d"}
Nov 03 16:11:15.041235 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:15.040851    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"648f1fa2-0c16-4abc-964b-200caa87def3","Type":"ContainerStarted","Data":"451a398d8f3701acc1e330e6b07220825a96f19d767faea578e8934b8214fe9c"}
Nov 03 16:11:15.041235 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:15.040875    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"648f1fa2-0c16-4abc-964b-200caa87def3","Type":"ContainerStarted","Data":"5d2b32299b44ca1da0d6f3eba1edf3c294922b05f8111791d121c6821c1f820e"}
Nov 03 16:11:15.041235 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:15.040900    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"648f1fa2-0c16-4abc-964b-200caa87def3","Type":"ContainerStarted","Data":"d1b66d21380757609c8e372278f99f484d85be96b036b16b4c2419b8483f9fd1"}
Nov 03 16:11:16.050132 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:16.050099    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"648f1fa2-0c16-4abc-964b-200caa87def3","Type":"ContainerStarted","Data":"3335ad2795e506275cfd2c0b9ca9ce547b94b5c534a5327b77fb6f3040898e29"}
Nov 03 16:11:16.296591 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:16.296494    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.979186929 podStartE2EDuration="7.296474532s" totalImagesPullingTime="5.306098407s" totalInitContainerRuntime="11.189196ms" isStatefulPod=false podCreationTimestamp="2026-11-03 16:11:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:11:16.245218379 +0000 UTC m=+99.077859061" watchObservedRunningTime="2026-11-03 16:11:16.296474532 +0000 UTC m=+99.129115206"
Nov 03 16:11:19.108530 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.108499    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-8b6c46946-f7klz"]
Nov 03 16:11:19.111566 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.111526    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.208145 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.208104    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8b6c46946-f7klz"]
Nov 03 16:11:19.235444 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.235414    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4djl6\" (UniqueName: \"kubernetes.io/projected/76785a65-390d-4367-ac2b-6bf096c6b18d-kube-api-access-4djl6\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.235606 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.235461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-service-ca\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.235606 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.235480    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76785a65-390d-4367-ac2b-6bf096c6b18d-tmp\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.235686 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.235606    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-trusted-ca-bundle\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.235686 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.235646    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-console-config\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.235777 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.235760    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-oauth-serving-cert\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.235836 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.235819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-oauth-config\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.235876 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.235847    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-serving-cert\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.336783 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.336748    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-oauth-config\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.336783 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.336784    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-serving-cert\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.337013 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.336805    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4djl6\" (UniqueName: \"kubernetes.io/projected/76785a65-390d-4367-ac2b-6bf096c6b18d-kube-api-access-4djl6\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.337013 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.336838    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-service-ca\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.337013 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.336852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76785a65-390d-4367-ac2b-6bf096c6b18d-tmp\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.337013 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.336872    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-trusted-ca-bundle\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.337013 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.336899    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-console-config\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.337013 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.336941    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-oauth-serving-cert\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.337450 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.337392    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76785a65-390d-4367-ac2b-6bf096c6b18d-tmp\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.337706 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.337682    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-service-ca\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.337879 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.337857    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-oauth-serving-cert\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.337931 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.337889    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-trusted-ca-bundle\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.338076 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.338051    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-console-config\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.339303 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.339281    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-oauth-config\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.339559 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.339540    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-serving-cert\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.398828 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.398749    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4djl6\" (UniqueName: \"kubernetes.io/projected/76785a65-390d-4367-ac2b-6bf096c6b18d-kube-api-access-4djl6\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.422640 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.422604    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:19.793219 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:11:19.793194    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76785a65_390d_4367_ac2b_6bf096c6b18d.slice/crio-99f565798e7ad8e99005e421011b93c887c108cb824c7635b668e019284a8056 WatchSource:0}: Error finding container 99f565798e7ad8e99005e421011b93c887c108cb824c7635b668e019284a8056: Status 404 returned error can't find the container with id 99f565798e7ad8e99005e421011b93c887c108cb824c7635b668e019284a8056
Nov 03 16:11:19.794332 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:19.794308    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8b6c46946-f7klz"]
Nov 03 16:11:20.067632 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:20.067549    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b6c46946-f7klz" event={"ID":"76785a65-390d-4367-ac2b-6bf096c6b18d","Type":"ContainerStarted","Data":"a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f"}
Nov 03 16:11:20.067632 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:20.067618    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b6c46946-f7klz" event={"ID":"76785a65-390d-4367-ac2b-6bf096c6b18d","Type":"ContainerStarted","Data":"99f565798e7ad8e99005e421011b93c887c108cb824c7635b668e019284a8056"}
Nov 03 16:11:20.191501 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:20.191443    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-8b6c46946-f7klz" podStartSLOduration=1.191426151 podStartE2EDuration="1.191426151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:11:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:11:20.190457941 +0000 UTC m=+103.023098613" watchObservedRunningTime="2026-11-03 16:11:20.191426151 +0000 UTC m=+103.024066823"
Nov 03 16:11:25.561548 ip-10-0-142-22 kubenswrapper[2582]: E1103 16:11:25.561506    2582 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:11:29.423105 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:29.423060    2582 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:29.423490 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:29.423121    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:29.427759 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:29.427743    2582 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:30.101490 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:30.101460    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:30.274787 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:30.274754    2582 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dd847f68b-wq7jc"]
Nov 03 16:11:47.570181 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.570127    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:11:47.570662 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.570198    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:11:47.570662 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.570231    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:47.623090 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.623058    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 16:11:47.674119 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.674088    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 16:11:47.674119 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.674096    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 16:11:47.683705 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.683681    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 16:11:47.684182 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.684160    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:47.684182 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.684174    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:11:47.693930 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.693907    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:11:47.801353 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.801322    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zsc2l\""
Nov 03 16:11:47.810033 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.809998    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:11:47.835890 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.835872    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rqr4v\""
Nov 03 16:11:47.844038 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.844020    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:11:47.849393 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.849183    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-j7zlt\""
Nov 03 16:11:47.857544 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:47.857339    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:48.183239 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:48.183201    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lqvng"]
Nov 03 16:11:48.212482 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:11:48.212451    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda67334f9_d13e_452c_b266_e0e996dd7db3.slice/crio-8f6f806a460f88a83e2b5fa8a4611360fa35f6a8872bcf20f755fe5e8e008d31 WatchSource:0}: Error finding container 8f6f806a460f88a83e2b5fa8a4611360fa35f6a8872bcf20f755fe5e8e008d31: Status 404 returned error can't find the container with id 8f6f806a460f88a83e2b5fa8a4611360fa35f6a8872bcf20f755fe5e8e008d31
Nov 03 16:11:48.225168 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:11:48.225143    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcffd0b63_ba73_433f_886f_fcae960b41a1.slice/crio-74f17e70dee7783d3a22e38518b3bbc7b7defb865b7675ab1d7f610bae1c17f3 WatchSource:0}: Error finding container 74f17e70dee7783d3a22e38518b3bbc7b7defb865b7675ab1d7f610bae1c17f3: Status 404 returned error can't find the container with id 74f17e70dee7783d3a22e38518b3bbc7b7defb865b7675ab1d7f610bae1c17f3
Nov 03 16:11:48.226704 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:48.226657    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k5rqd"]
Nov 03 16:11:48.271964 ip-10-0-142-22 kubenswrapper[2582]: W1103 16:11:48.271931    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f6992c_48b9_4bb5_9b85_a37552b64c8b.slice/crio-850880eb67311ce34cb01c81fdbbf517468eeaad7bf75d1fd085be358f4029b1 WatchSource:0}: Error finding container 850880eb67311ce34cb01c81fdbbf517468eeaad7bf75d1fd085be358f4029b1: Status 404 returned error can't find the container with id 850880eb67311ce34cb01c81fdbbf517468eeaad7bf75d1fd085be358f4029b1
Nov 03 16:11:48.273206 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:48.273182    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tbpr9"]
Nov 03 16:11:49.158561 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:49.158522    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tbpr9" event={"ID":"c6f6992c-48b9-4bb5-9b85-a37552b64c8b","Type":"ContainerStarted","Data":"850880eb67311ce34cb01c81fdbbf517468eeaad7bf75d1fd085be358f4029b1"}
Nov 03 16:11:49.159691 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:49.159665    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lqvng" event={"ID":"a67334f9-d13e-452c-b266-e0e996dd7db3","Type":"ContainerStarted","Data":"8f6f806a460f88a83e2b5fa8a4611360fa35f6a8872bcf20f755fe5e8e008d31"}
Nov 03 16:11:49.160885 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:49.160854    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k5rqd" event={"ID":"cffd0b63-ba73-433f-886f-fcae960b41a1","Type":"ContainerStarted","Data":"74f17e70dee7783d3a22e38518b3bbc7b7defb865b7675ab1d7f610bae1c17f3"}
Nov 03 16:11:50.165834 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:50.165794    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k5rqd" event={"ID":"cffd0b63-ba73-433f-886f-fcae960b41a1","Type":"ContainerStarted","Data":"d14d6eb20c9c53ea43dc7263c0d4273b979b456e6129e2c756843dddf83ee79b"}
Nov 03 16:11:50.166265 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:50.165896    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k5rqd" event={"ID":"cffd0b63-ba73-433f-886f-fcae960b41a1","Type":"ContainerStarted","Data":"67883b713802abf3a99bc9d36f77abe0601b45eaf007f34b520d9f13452632c1"}
Nov 03 16:11:50.287862 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:50.287808    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k5rqd" podStartSLOduration=130.762305129 podStartE2EDuration="2m12.28779377s" totalImagesPullingTime="1.525488641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:11:50.28684182 +0000 UTC m=+133.119482508" watchObservedRunningTime="2026-11-03 16:11:50.28779377 +0000 UTC m=+133.120434441"
Nov 03 16:11:53.179247 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:53.179160    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lqvng" event={"ID":"a67334f9-d13e-452c-b266-e0e996dd7db3","Type":"ContainerStarted","Data":"d8dd845f93e2ccf289a14961fcdffe503f46076abc60f3253582b884081ca470"}
Nov 03 16:11:53.179836 ip-10-0-142-22 kubenswrapper[2582]: I1103 16:11:53.179436    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:11:54.109627 ip-10-0-142-22 systemd[1]: Stopping Kubernetes Kubelet...
Nov 03 16:11:54.118337 ip-10-0-142-22 systemd[1]: kubelet.service: Deactivated successfully.
Nov 03 16:11:54.118505 ip-10-0-142-22 systemd[1]: Stopped Kubernetes Kubelet.
Nov 03 16:11:54.118681 ip-10-0-142-22 systemd[1]: kubelet.service: Consumed 5.952s CPU time.
Nov 03 16:11:54.126406 ip-10-0-142-22 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 16:11:54.210126 ip-10-0-142-22 kubenswrapper[5605]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 16:11:54.210126 ip-10-0-142-22 kubenswrapper[5605]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 16:11:54.210126 ip-10-0-142-22 kubenswrapper[5605]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 16:11:54.210126 ip-10-0-142-22 kubenswrapper[5605]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216320    5605 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216412    5605 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216416    5605 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216419    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216422    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216425    5605 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216428    5605 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216431    5605 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216434    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216437    5605 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:11:54.216421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216440    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216445    5605 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216451    5605 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216454    5605 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216457    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216460    5605 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216463    5605 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216466    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216469    5605 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216471    5605 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216474    5605 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216476    5605 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216479    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216482    5605 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216485    5605 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216488    5605 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216490    5605 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216494    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216497    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:11:54.217059 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216500    5605 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216506    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216509    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216512    5605 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216514    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216517    5605 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216519    5605 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216522    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216526    5605 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216528    5605 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216531    5605 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216534    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216536    5605 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216540    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216544    5605 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216546    5605 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216549    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216552    5605 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216555    5605 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:11:54.217911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216557    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216560    5605 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216564    5605 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216567    5605 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216569    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216572    5605 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216574    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216577    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216579    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216582    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216584    5605 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216587    5605 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216590    5605 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216593    5605 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216596    5605 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216598    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216601    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216603    5605 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216606    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:11:54.218808 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216608    5605 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216611    5605 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216613    5605 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216616    5605 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216619    5605 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216626    5605 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216628    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216631    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216633    5605 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216636    5605 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216639    5605 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216641    5605 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216644    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216647    5605 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216649    5605 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216652    5605 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216655    5605 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216658    5605 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216661    5605 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216663    5605 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216666    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:11:54.219614 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216668    5605 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216671    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216673    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216676    5605 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216681    5605 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216685    5605 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216688    5605 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216691    5605 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216693    5605 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216696    5605 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216698    5605 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216701    5605 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216704    5605 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216706    5605 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216709    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216711    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216717    5605 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.216719    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217560    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:11:54.220463 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217569    5605 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217573    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217576    5605 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217580    5605 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217583    5605 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217585    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217588    5605 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217590    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217595    5605 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217599    5605 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217603    5605 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217606    5605 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217608    5605 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217611    5605 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217614    5605 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217616    5605 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217620    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217623    5605 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217625    5605 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:11:54.221265 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217628    5605 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217631    5605 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217633    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217636    5605 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217638    5605 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217641    5605 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217644    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217647    5605 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217649    5605 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217652    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217656    5605 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217659    5605 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217661    5605 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217664    5605 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217667    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217669    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217672    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217674    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217677    5605 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:11:54.221856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217681    5605 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217684    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217688    5605 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217691    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217694    5605 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217697    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217700    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217703    5605 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217705    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217708    5605 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217711    5605 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217713    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217716    5605 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217718    5605 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217721    5605 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217742    5605 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217745    5605 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217747    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217750    5605 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:11:54.222350 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217753    5605 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217756    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217758    5605 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217761    5605 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217765    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217768    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217770    5605 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217773    5605 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217776    5605 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217778    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217781    5605 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217783    5605 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217786    5605 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217788    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217791    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217794    5605 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217797    5605 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217800    5605 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217804    5605 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217807    5605 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:11:54.222854 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217810    5605 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217813    5605 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217815    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217818    5605 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217821    5605 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217823    5605 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217826    5605 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217828    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217831    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217834    5605 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217836    5605 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217839    5605 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217841    5605 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217844    5605 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217846    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217850    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217854    5605 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217856    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217859    5605 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217861    5605 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:11:54.223370 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217864    5605 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217867    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217869    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217872    5605 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217874    5605 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217877    5605 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217879    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.217882    5605 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218386    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218391    5605 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218394    5605 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218397    5605 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218400    5605 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218402    5605 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218405    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218408    5605 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218410    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218413    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218416    5605 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218418    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:11:54.223893 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218421    5605 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218423    5605 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218426    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218430    5605 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218432    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218435    5605 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218437    5605 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218440    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218445    5605 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218448    5605 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218451    5605 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218454    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218456    5605 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218459    5605 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218462    5605 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218464    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218467    5605 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218470    5605 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218472    5605 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218475    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:11:54.224409 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218477    5605 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218480    5605 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218482    5605 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218488    5605 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218491    5605 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218494    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218496    5605 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218499    5605 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218502    5605 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218504    5605 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218507    5605 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218509    5605 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218512    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218514    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218517    5605 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218519    5605 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218522    5605 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218524    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218527    5605 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218530    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:11:54.224909 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218534    5605 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218537    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218539    5605 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218544    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218546    5605 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218549    5605 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218552    5605 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218554    5605 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218557    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218559    5605 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218562    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218565    5605 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218567    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218571    5605 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218575    5605 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218578    5605 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218584    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218587    5605 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218590    5605 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:11:54.225440 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218593    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218595    5605 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218598    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218600    5605 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218603    5605 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218605    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218608    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218611    5605 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218613    5605 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218616    5605 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218618    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218621    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218623    5605 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218627    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218630    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218632    5605 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218636    5605 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218639    5605 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218641    5605 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218644    5605 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:11:54.226141 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218646    5605 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218649    5605 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218652    5605 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218654    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218657    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218659    5605 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218662    5605 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218664    5605 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218667    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218669    5605 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218673    5605 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218675    5605 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218678    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218682    5605 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.218685    5605 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218809    5605 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218827    5605 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218834    5605 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218838    5605 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218843    5605 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218846    5605 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 16:11:54.226634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218850    5605 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218855    5605 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218858    5605 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218861    5605 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218865    5605 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218869    5605 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218872    5605 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218876    5605 flags.go:64] FLAG: --cgroup-root=""
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218880    5605 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218883    5605 flags.go:64] FLAG: --client-ca-file=""
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218885    5605 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218888    5605 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218892    5605 flags.go:64] FLAG: --cluster-domain=""
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218895    5605 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218898    5605 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218901    5605 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218905    5605 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218908    5605 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218912    5605 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218915    5605 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218918    5605 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218921    5605 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218926    5605 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218929    5605 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 16:11:54.227476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218932    5605 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218935    5605 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218939    5605 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218943    5605 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218945    5605 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218949    5605 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218952    5605 flags.go:64] FLAG: --enable-server="true"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218954    5605 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218959    5605 flags.go:64] FLAG: --event-burst="100"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218962    5605 flags.go:64] FLAG: --event-qps="50"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218965    5605 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218968    5605 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218973    5605 flags.go:64] FLAG: --eviction-hard=""
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218977    5605 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218980    5605 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218984    5605 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218987    5605 flags.go:64] FLAG: --eviction-soft=""
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218990    5605 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218993    5605 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218996    5605 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.218999    5605 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219001    5605 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219004    5605 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219007    5605 flags.go:64] FLAG: --feature-gates=""
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219011    5605 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 16:11:54.228431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219014    5605 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219017    5605 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219020    5605 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219023    5605 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219026    5605 flags.go:64] FLAG: --help="false"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219029    5605 flags.go:64] FLAG: --hostname-override="ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219033    5605 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219036    5605 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219039    5605 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219042    5605 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219045    5605 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219048    5605 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219051    5605 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219054    5605 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219057    5605 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219060    5605 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219063    5605 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219066    5605 flags.go:64] FLAG: --kube-reserved=""
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219068    5605 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219073    5605 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219076    5605 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219079    5605 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219083    5605 flags.go:64] FLAG: --lock-file=""
Nov 03 16:11:54.229433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219086    5605 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219089    5605 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219092    5605 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219098    5605 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219101    5605 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219104    5605 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219106    5605 flags.go:64] FLAG: --logging-format="text"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219109    5605 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219113    5605 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219115    5605 flags.go:64] FLAG: --manifest-url=""
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219118    5605 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219123    5605 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219126    5605 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219131    5605 flags.go:64] FLAG: --max-pods="110"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219143    5605 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219148    5605 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219153    5605 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219158    5605 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219163    5605 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219167    5605 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219171    5605 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219181    5605 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219186    5605 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219191    5605 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219195    5605 flags.go:64] FLAG: --pod-cidr=""
Nov 03 16:11:54.230599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219200    5605 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219204    5605 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219209    5605 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219215    5605 flags.go:64] FLAG: --port="10250"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219220    5605 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219224    5605 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0442dc01194780f5f"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219230    5605 flags.go:64] FLAG: --qos-reserved=""
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219235    5605 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219240    5605 flags.go:64] FLAG: --register-node="true"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219244    5605 flags.go:64] FLAG: --register-with-taints=""
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219250    5605 flags.go:64] FLAG: --registry-burst="10"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219254    5605 flags.go:64] FLAG: --registry-qps="5"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219259    5605 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219263    5605 flags.go:64] FLAG: --reserved-memory=""
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219269    5605 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219274    5605 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219279    5605 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219283    5605 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219288    5605 flags.go:64] FLAG: --runonce="false"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219292    5605 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219297    5605 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219301    5605 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219306    5605 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219310    5605 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219316    5605 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219320    5605 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 16:11:54.231738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219325    5605 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219329    5605 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219334    5605 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219339    5605 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219343    5605 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219348    5605 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219352    5605 flags.go:64] FLAG: --system-cgroups=""
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219356    5605 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219365    5605 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219371    5605 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219375    5605 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219382    5605 flags.go:64] FLAG: --tls-min-version=""
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219388    5605 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219393    5605 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219397    5605 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219402    5605 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219407    5605 flags.go:64] FLAG: --v="2"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219413    5605 flags.go:64] FLAG: --version="false"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219420    5605 flags.go:64] FLAG: --vmodule=""
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219426    5605 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.219431    5605 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219674    5605 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219680    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219686    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:11:54.232882 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219692    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219697    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219702    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219707    5605 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219712    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219716    5605 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219740    5605 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219746    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219750    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219755    5605 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219759    5605 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219763    5605 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219767    5605 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219771    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219776    5605 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219780    5605 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219784    5605 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219790    5605 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219795    5605 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219799    5605 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:11:54.233932 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219805    5605 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219810    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219814    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219818    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219823    5605 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219827    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219831    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219836    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219840    5605 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219844    5605 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219848    5605 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219852    5605 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219856    5605 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219861    5605 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219865    5605 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219869    5605 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219873    5605 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219877    5605 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219881    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:11:54.234804 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219886    5605 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219890    5605 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219895    5605 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219899    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219904    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219908    5605 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219912    5605 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219917    5605 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219921    5605 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219925    5605 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219930    5605 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219934    5605 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219938    5605 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219944    5605 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219948    5605 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219953    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219957    5605 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219961    5605 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219966    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219970    5605 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:11:54.235650 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219974    5605 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219979    5605 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219983    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219987    5605 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219991    5605 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219995    5605 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.219999    5605 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220003    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220007    5605 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220014    5605 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220019    5605 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220023    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220030    5605 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220036    5605 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220042    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220046    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220050    5605 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220054    5605 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220058    5605 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:11:54.236799 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220063    5605 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220067    5605 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220071    5605 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220079    5605 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220083    5605 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220088    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220094    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220099    5605 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220103    5605 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220107    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220111    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220115    5605 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220120    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220124    5605 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220129    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220133    5605 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220137    5605 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220141    5605 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220145    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220149    5605 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:11:54.237616 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220154    5605 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220158    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220162    5605 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220166    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220171    5605 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.220212    5605 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220269    5605 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220275    5605 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220281    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220286    5605 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220290    5605 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220295    5605 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220299    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220303    5605 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220307    5605 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220311    5605 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:11:54.238422 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220316    5605 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220321    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220327    5605 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220333    5605 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220339    5605 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220344    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220350    5605 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220356    5605 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220363    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220368    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220372    5605 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220376    5605 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220381    5605 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220385    5605 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220389    5605 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220394    5605 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220398    5605 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220402    5605 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220406    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:11:54.239107 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220411    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220415    5605 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220420    5605 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220424    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220429    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220433    5605 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220438    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220443    5605 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220447    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220452    5605 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220456    5605 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220460    5605 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220464    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220470    5605 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220474    5605 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220482    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220487    5605 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220492    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220496    5605 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:11:54.239885 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220500    5605 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220505    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220509    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220513    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220518    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220523    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220527    5605 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220532    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220536    5605 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220542    5605 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220546    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220551    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220555    5605 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220559    5605 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220563    5605 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220568    5605 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220572    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220576    5605 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220580    5605 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220585    5605 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:11:54.240530 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220590    5605 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220595    5605 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220599    5605 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220604    5605 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220609    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220613    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220618    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220623    5605 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220629    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220633    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220638    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220642    5605 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220647    5605 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220651    5605 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220655    5605 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220659    5605 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220664    5605 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220668    5605 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220672    5605 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:11:54.241505 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220676    5605 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220681    5605 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220685    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220689    5605 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220693    5605 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220698    5605 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220703    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220708    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220712    5605 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220716    5605 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220720    5605 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220741    5605 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220746    5605 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220750    5605 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220756    5605 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220760    5605 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220765    5605 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220769    5605 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.220773    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.225109    5605 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 16:11:54.242354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.225124    5605 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.225143    5605 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.225148    5605 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225224    5605 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225230    5605 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225234    5605 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225239    5605 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225243    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225246    5605 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225249    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225251    5605 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225254    5605 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225257    5605 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225259    5605 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225262    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225265    5605 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225267    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225270    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:11:54.243276 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225272    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225275    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225278    5605 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225282    5605 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225286    5605 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225289    5605 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225292    5605 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225294    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225298    5605 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225300    5605 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225303    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225306    5605 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225308    5605 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225311    5605 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225313    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225316    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225319    5605 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225321    5605 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225324    5605 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:11:54.244085 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225326    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225328    5605 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225331    5605 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225334    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225336    5605 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225339    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225341    5605 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225344    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225346    5605 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225349    5605 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225351    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225354    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225356    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225358    5605 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225361    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225363    5605 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225366    5605 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225368    5605 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225371    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225373    5605 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:11:54.244930 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225376    5605 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225379    5605 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225383    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225386    5605 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225388    5605 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225390    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225393    5605 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225395    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225398    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225400    5605 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225403    5605 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225405    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225408    5605 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225411    5605 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225414    5605 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225417    5605 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225419    5605 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225422    5605 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225424    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225427    5605 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:11:54.245848 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225429    5605 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225432    5605 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225434    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225437    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225439    5605 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225442    5605 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225445    5605 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225448    5605 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225450    5605 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225453    5605 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225455    5605 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225458    5605 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225461    5605 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225464    5605 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225466    5605 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225475    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225478    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225481    5605 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225483    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225486    5605 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:11:54.246704 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225488    5605 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225491    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225493    5605 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225496    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225499    5605 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225501    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225503    5605 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225506    5605 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225509    5605 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225511    5605 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225514    5605 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225516    5605 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.225544    5605 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225761    5605 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225766    5605 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225769    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:11:54.247405 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225772    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225775    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225778    5605 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225781    5605 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225783    5605 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225786    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225788    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225791    5605 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225794    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225796    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225799    5605 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225801    5605 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225809    5605 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225812    5605 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225815    5605 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225818    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225821    5605 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225823    5605 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225825    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:11:54.247863 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225828    5605 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225832    5605 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225836    5605 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225839    5605 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225842    5605 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225845    5605 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225847    5605 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225849    5605 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225852    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225854    5605 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225857    5605 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225859    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225862    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225864    5605 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225866    5605 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225869    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225871    5605 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225874    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225876    5605 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:11:54.248359 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225879    5605 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225881    5605 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225884    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225887    5605 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225889    5605 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225892    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225894    5605 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225902    5605 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225905    5605 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225908    5605 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225910    5605 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225913    5605 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225915    5605 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225918    5605 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225921    5605 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225923    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225926    5605 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225929    5605 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225932    5605 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225936    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:11:54.248894 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225938    5605 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225941    5605 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225944    5605 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225946    5605 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225949    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225951    5605 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225954    5605 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225956    5605 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225959    5605 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225961    5605 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225964    5605 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225966    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225968    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225971    5605 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225974    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225977    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225979    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225982    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225985    5605 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225987    5605 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:11:54.249705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.225996    5605 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226002    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226004    5605 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226007    5605 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226010    5605 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226012    5605 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226015    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226017    5605 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226020    5605 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226022    5605 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226025    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226027    5605 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226030    5605 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226032    5605 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226035    5605 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226037    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226040    5605 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226043    5605 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226045    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226047    5605 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:11:54.250596 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226050    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226052    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226055    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226057    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226060    5605 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.226084    5605 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226128    5605 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226132    5605 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226137    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226140    5605 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226142    5605 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226145    5605 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226148    5605 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226157    5605 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226160    5605 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226163    5605 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 16:11:54.251499 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226165    5605 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226168    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226171    5605 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226173    5605 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226176    5605 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226178    5605 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226180    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226183    5605 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226185    5605 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226188    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226190    5605 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226193    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226195    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226198    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226200    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226203    5605 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226206    5605 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226211    5605 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226214    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 16:11:54.252332 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226217    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226220    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226222    5605 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226225    5605 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226227    5605 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226232    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226234    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226237    5605 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226239    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226242    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226245    5605 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226253    5605 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226256    5605 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226259    5605 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226261    5605 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226264    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226267    5605 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226269    5605 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226272    5605 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 16:11:54.253282 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226274    5605 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226278    5605 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226282    5605 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226285    5605 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226288    5605 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226291    5605 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226293    5605 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226296    5605 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226298    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226300    5605 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226303    5605 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226305    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226308    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226310    5605 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226313    5605 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226315    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226323    5605 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226326    5605 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226330    5605 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 16:11:54.254121 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226333    5605 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226336    5605 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226338    5605 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226341    5605 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226345    5605 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226347    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226355    5605 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226358    5605 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226361    5605 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226363    5605 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226366    5605 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226368    5605 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226371    5605 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226373    5605 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226376    5605 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226379    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226381    5605 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226386    5605 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226390    5605 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226392    5605 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 16:11:54.254979 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226395    5605 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226398    5605 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226400    5605 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226403    5605 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226405    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226408    5605 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226410    5605 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226413    5605 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226415    5605 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226418    5605 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226420    5605 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226424    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226427    5605 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226430    5605 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226432    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226435    5605 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226439    5605 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226442    5605 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:11:54.226449    5605 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.226508    5605 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 16:11:54.255771 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.228001    5605 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.228091    5605 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.228396    5605 server.go:1014] "Starting client certificate rotation"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.228569    5605 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.228642    5605 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 16:04:38 +0000 UTC" deadline="2028-05-17 10:06:57.642098011 +0000 UTC"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.228693    5605 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13457h55m3.413407472s"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.229425    5605 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.230776    5605 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.234386    5605 log.go:25] "Validated CRI v1 runtime API"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.236281    5605 log.go:25] "Validated CRI v1 image API"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.236993    5605 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 16:11:54.256551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.239987    5605 fs.go:135] Filesystem UUIDs: map[4db0fe6b-a6a2-4b34-975a-22cd5ccc7aca:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bfe799c6-b6be-4cc8-83db-43b8a44f5da5:/dev/nvme0n1p3]
Nov 03 16:11:54.257075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.240011    5605 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/29dad769d3401de44406bfe76091c3caa1c312098ec7c27d72c89c748eec01d8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/29dad769d3401de44406bfe76091c3caa1c312098ec7c27d72c89c748eec01d8/userdata/shm major:0 minor:89 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2e2a0550fd2598e68d0ca1f4731c713f02134f627850a60044638e380f61fc2b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2e2a0550fd2598e68d0ca1f4731c713f02134f627850a60044638e380f61fc2b/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3019d8ce3631c97e275f3646298176c2f6355d9291eb32d4ada3fd0e87496b55/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3019d8ce3631c97e275f3646298176c2f6355d9291eb32d4ada3fd0e87496b55/userdata/shm major:0 minor:303 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4d67fc0bee521b1a0c4198f96b87ad9bc5ceea29046fbb617ad8e4076d91d853/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4d67fc0bee521b1a0c4198f96b87ad9bc5ceea29046fbb617ad8e4076d91d853/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/52fa4ae03cad2bfea8518e5e02e5c89a84500eaf6176c3682eabf55b08d4df93/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/52fa4ae03cad2bfea8518e5e02e5c89a84500eaf6176c3682eabf55b08d4df93/userdata/shm major:0 minor:83 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/54997c0f21445fa127f7c31027a61b9a9c73aeb4b577450da384c4c885c5d6f7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/54997c0f21445fa127f7c31027a61b9a9c73aeb4b577450da384c4c885c5d6f7/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6a15c0e001c60f152152a613b3c0365ac1f8c5032b7af7caadb111e177f8929c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6a15c0e001c60f152152a613b3c0365ac1f8c5032b7af7caadb111e177f8929c/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/74f17e70dee7783d3a22e38518b3bbc7b7defb865b7675ab1d7f610bae1c17f3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/74f17e70dee7783d3a22e38518b3bbc7b7defb865b7675ab1d7f610bae1c17f3/userdata/shm major:0 minor:361 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7f6d93c7656545212da5ac11e453f70ab88ee42e872c14e6c466cd8dbe78ad2c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7f6d93c7656545212da5ac11e453f70ab88ee42e872c14e6c466cd8dbe78ad2c/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/839386ca47a23cf94d69213f6f6395de4fa713f3bfb6dd4b5192343ab1c0c969/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/839386ca47a23cf94d69213f6f6395de4fa713f3bfb6dd4b5192343ab1c0c969/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/850880eb67311ce34cb01c81fdbbf517468eeaad7bf75d1fd085be358f4029b1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/850880eb67311ce34cb01c81fdbbf517468eeaad7bf75d1fd085be358f4029b1/userdata/shm major:0 minor:363 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8f6f806a460f88a83e2b5fa8a4611360fa35f6a8872bcf20f755fe5e8e008d31/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8f6f806a460f88a83e2b5fa8a4611360fa35f6a8872bcf20f755fe5e8e008d31/userdata/shm major:0 minor:359 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/92821c6fc58b809f6cd483310cd470578eb71531fc5f8dce6914f49fa9b015c1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/92821c6fc58b809f6cd483310cd470578eb71531fc5f8dce6914f49fa9b015c1/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/987d64b915ab5b9587adb18f4ec8af3d3d9179036bd21512ceebd2aa91a707fd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/987d64b915ab5b9587adb18f4ec8af3d3d9179036bd21512ceebd2aa91a707fd/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/99f565798e7ad8e99005e421011b93c887c108cb824c7635b668e019284a8056/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/99f565798e7ad8e99005e421011b93c887c108cb824c7635b668e019284a8056/userdata/shm major:0 minor:346 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/af1e5abda95474af2c5ae13d8bfbe3d225bd8c2911b774b657c401c97b3f0534/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/af1e5abda95474af2c5ae13d8bfbe3d225bd8c2911b774b657c401c97b3f0534/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eb0e217e600d3bdca1e147cbb5263225a859fa5159e34f9499c07b7567c1396c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eb0e217e600d3bdca1e147cbb5263225a859fa5159e34f9499c07b7567c1396c/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/edbb68bb5cbcfaf8e36384356bc1f636bfc4922eaebe1e7d75ae041fd7ca91db/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/edbb68bb5cbcfaf8e36384356bc1f636bfc4922eaebe1e7d75ae041fd7ca91db/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f5a5565102010c4d5acf9f84f18c149ae4f0c16d9461cddbc8ff8e6bf14f7795/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f5a5565102010c4d5acf9f84f18c149ae4f0c16d9461cddbc8ff8e6bf14f7795/userdata/shm major:0 minor:284 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f78dc792b42754170fd76c386076ebb557d32b1e906205156736d45231680942/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f78dc792b42754170fd76c386076ebb557d32b1e906205156736d45231680942/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f88d78bcdab59ab026d9e8b15d0f7dbd94b3b631627ecd7576c5dcbb205ab3f6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f88d78bcdab59ab026d9e8b15d0f7dbd94b3b631627ecd7576c5dcbb205ab3f6/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0599707f-1cdb-4e9f-99da-72703b028198/volumes/kubernetes.io~projected/kube-api-access-r7nln:{mountpoint:/var/lib/kubelet/pods/0599707f-1cdb-4e9f-99da-72703b028198/volumes/kubernetes.io~projected/kube-api-access-r7nln major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1897a95d-8237-4a69-9ae8-c229490ba88c/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/1897a95d-8237-4a69-9ae8-c229490ba88c/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1897a95d-8237-4a69-9ae8-c229490ba88c/volumes/kubernetes.io~projected/kube-api-access-k4pnj:{mountpoint:/var/lib/kubelet/pods/1897a95d-8237-4a69-9ae8-c229490ba88c/volumes/kubernetes.io~projected/kube-api-access-k4pnj major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1897a95d-8237-4a69-9ae8-c229490ba88c/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/1897a95d-8237-4a69-9ae8-c229490ba88c/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c560131-5ae7-42ef-975b-23af6909fb89/volumes/kubernetes.io~projected/kube-api-access-c7l95:{mountpoint:/var/lib/kubelet/pods/1c560131-5ae7-42ef-975b-23af6909fb89/volumes/kubernetes.io~projected/kube-api-access-c7l95 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2cfc5077-0c47-48fd-a247-51156f01935c/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/2cfc5077-0c47-48fd-a247-51156f01935c/volumes/kubernetes.io~secret/agent-certs major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/38701b60-1908-4568-81ed-0c5a91716678/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/38701b60-1908-4568-81ed-0c5a91716678/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/38701b60-1908-4568-81ed-0c5a91716678/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/38701b60-1908-4568-81ed-0c5a91716678/volumes/kubernetes.io~empty-dir/tmp major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/38701b60-1908-4568-81ed-0c5a91716678/volumes/kubernetes.io~projected/kube-api-access-sfl52:{mountpoint:/var/lib/kubelet/pods/38701b60-1908-4568-81ed-0c5a91716678/volumes/kubernetes.io~projected/kube-api-access-sfl52 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/54d61260-b3aa-46f1-8cdd-d15102a208ce/volumes/kubernetes.io~projected/kube-api-access-wzk2k:{mountpoint:/var/lib/kubelet/pods/54d61260-b3aa-46f1-8cdd-d15102a208ce/volumes/kubernetes.io~projected/kube-api-access-wzk2k major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/54d61260-b3aa-46f1-8cdd-d15102a208ce/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/54d61260-b3aa-46f1-8cdd-d15102a208ce/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/web-config/alertmanager/9 major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/web-config/config-reloader/6 major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~empty-dir/config-out major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~projected/kube-api-access-ddv4f:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~projected/kube-api-access-ddv4f major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~projected/tls-assets major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/config-volume major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:298 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/web-config major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76785a65-390d-4367-ac2b-6bf096c6b18d/volumes/kubernetes.io~projected/kube-api-access-4djl6:{mountpoint:/var/lib/kubelet/pods/76785a65-390d-4367-ac2b-6bf096c6b18d/volumes/kubernetes.io~projected/kube-api-access-4djl6 major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76785a65-390d-4367-ac2b-6bf096c6b18d/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/76785a65-390d-4367-ac2b-6bf096c6b18d/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:340 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76785a65-390d-4367-ac2b-6bf096c6b18d/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/76785a65-390d-4367-ac2b-6bf096c6b18d/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8ad82d3f-4298-4eb1-979b-91f94c007cef/volumes/kubernetes.io~projected/kube-api-access-hh6ld:{mountpoint:/var/lib/kubelet/pods/8ad82d3f-4298-4eb1-979b-91f94c007cef/volumes/kubernetes.io~projected/kube-api-access-hh6ld major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8ad82d3f-4298-4eb1-979b-91f94c007cef/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/8ad82d3f-4298-4eb1-979b-91f94c007cef/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8ad82d3f-4298-4eb1-979b-91f94c007cef/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/8ad82d3f-4298-4eb1-979b-91f94c007cef/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a67334f9-d13e-452c-b266-e0e996dd7db3/volumes/kubernetes.io~projected/kube-api-access-zsxr9:{mountpoint:/var/lib/kubelet/pods/a67334f9-d13e-452c-b266-e0e996dd7db3/volumes/kubernetes.io~projected/kube-api-access-zsxr9 major:0 minor:358 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bafe6e21-68bb-4c42-84bf-96e38b7c1e55/volumes/kubernetes.io~projected/kube-api-access-mbshk:{mountpoint:/var/lib/kubelet/pods/bafe6e21-68bb-4c42-84bf-96e38b7c1e55/volumes/kubernetes.io~projected/kube-api-access-mbshk major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bafe6e21-68bb-4c42-84bf-96e38b7c1e55/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/bafe6e21-68bb-4c42-84bf-96e38b7c1e55/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bafe6e21-68bb-4c42-84bf-96e38b7c1e55/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/bafe6e21-68bb-4c42-84bf-96e38b7c1e55/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c14a9e56-182c-4abc-aa30-84b7958f775d/volumes/kubernetes.io~projected/kube-api-access-4brmn:{mountpoint:/var/lib/kubelet/pods/c14a9e56-182c-4abc-aa30-84b7958f775d/volumes/kubernetes.io~projected/kube-api-access-4brmn major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c6f6992c-48b9-4bb5-9b85-a37552b64c8b/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/c6f6992c-48b9-4bb5-9b85-a37552b64c8b/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c9693f80-8f98-41c3-a72e-089d647d47a2/volumes/kubernetes.io~projected/kube-api-access-zwx5m:{mountpoint:/var/lib/kubelet/pods/c9693f80-8f98-41c3-a72e-089d647d47a2/volumes/kubernetes.io~projected/kube-api-access-zwx5m major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c9693f80-8f98-41c3-a72e-089d647d47a2/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/c9693f80-8f98-41c3-a72e-089d647d47a2/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c9693f80-8f98-41c3-a72e-089d647d47a2/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/c9693f80-8f98-41c3-a72e-089d647d47a2/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cffd0b63-ba73-433f-886f-fcae960b41a1/volumes/kubernetes.io~projected/kube-api-access-sgqtp:{mountpoint:/var/lib/kubelet/pods/cffd0b63-ba73-433f-886f-fcae960b41a1/volumes/kubernetes.io~projected/kube-api-access-sgqtp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cffd0b63-ba73-433f-886f-fcae960b41a1/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/cffd0b63-ba73-433f-886f-fcae960b41a1/volumes/kubernetes.io~secret/metrics-certs major:0 minor:352 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3a76124-dbf8-4fb3-bc58-52a21819b9d9/volumes/kubernetes.io~projected/kube-api-access-z56b7:{mountpoint:/var/lib/kubelet/pods/d3a76124-dbf8-4fb3-bc58-52a21819b9d9/volumes/kubernetes.io~projected/kube-api-access-z56b7 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3a76124-dbf8-4fb3-bc58-52a21819b9d9/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/d3a76124-dbf8-4fb3-bc58-52a21819b9d9/volumes/kubernetes.io~secret/cert major:0 minor:128 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d80b9f00-54a1-4809-9856-efb0ee40bd24/volumes/kubernetes.io~projected/kube-api-access-sjvvq:{mountpoint:/var/lib/kubelet/pods/d80b9f00-54a1-4809-9856-efb0ee40bd24/volumes/kubernetes.io~projected/kube-api-access-sjvvq major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dadfa10c-198c-4f93-869f-377048834934/volumes/kubernetes.io~projected/kube-api-access-pl446:{mountpoint:/var/lib/kubelet/pods/dadfa10c-198c-4f93-869f-377048834934/volumes/kubernetes.io~projected/kube-api-access-pl446 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f239d145-6b28-421b-a024-e86135e1fb4d/volumes/kubernetes.io~projected/kube-api-access-kw9c7:{mountpoint:/var/lib/kubelet/pods/f239d145-6b28-421b-a024-e86135e1fb4d/volumes/kubernetes.io~projected/kube-api-access-kw9c7 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ffeba40d-6416-4f6f-b645-d6e3338f7adb/volumes/kubernetes.io~projected/kube-api-access-jl42j:{mountpoint:/var/lib/kubelet/pods/ffeba40d-6416-4f6f-b645-d6e3338f7adb/volumes/kubernetes.io~projected/kube-api-access-jl42j major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ffeba40d-6416-4f6f-b645-d6e3338f7adb/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/ffeba40d-6416-4f6f-b645-d6e3338f7adb/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:200 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/8a5cfe911b0856e70f9c4ac46477305770c8d573c695d3db5c985e8d73ac64a9/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/17e7421b184e010f201798d02a620ee26820d5184997ce94ff122e94e32c4291/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/2ba4c44463b8fcc3f94b8dd8956172ef81e8616f9166c5957606990261b5e264/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/0dcbe1a61dd4f4d4f2623155bb77f18e7cbc307cc470d323ad1ccfc7639d8291/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/430f8158eea3a235173d2ebbbcb8a3129c1c21808d6493d093378a9cb3937079/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/42580e35b1853c1412d683a797caff91ebd88c39a5eab90e8801bd270fe38c1b/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-122:{mountpoint:/var/lib/containers/storage/overlay/de256393eb2cc76998bb5ef80aa3e37bfe9240ecd6d944f54a55b9d4f01972a7/merged major:0 minor:122 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/0ab68c9d087367c980293638af3e3fdd86da125059b37d391ca5b843b945b50c/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/0b5d7dcb52f086b56740073f485e1275adc2c71b28de3ac58ca3f5bcac6d05e3/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/a1bb9a16a232d4926289e6dc49936c1b0d808deba3240eeb1fa0c710fd1c60da/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/09778f94d1d076729fc7019231b533b4b2492fc82181196fe702b633cb384559/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/8bc44e70d1a119826725432b0dc9e59fa5c43970cf08bf0ce185ecdd02417a6a/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/7c4db92906d4ac93034af7ff2d21a91e9c3fffa3f8b44783917c07f53d36e4ec/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-161:{mountpoint:/var/lib/containers/storage/overlay/e6c6ef34690d28ba2c3c9e1af896751d22bcadc86e487bf64ab64659bb2631ed/merged major:0 minor:161 fsType:overlay blockSize:0} overlay_0-166:{mountpoint:/var/lib/containers/storage/overlay/c442b758024b3c448a1f99723aff0e8892dd1908993f1dd9fffd5efbd8cafb14/merged major:0 minor:166 fsType:overlay blockSize:0} overlay_0-171:{mountpoint:/var/lib/containers/storage/overlay/29af39a3ff9e2fb55ebe5dd029bb311ffdc08d4315773bd2611471b8a7950e28/merged major:0 minor:171 fsType:overlay blockSize:0} overlay_0-176:{mountpoint:/var/lib/containers/storage/overlay/e74036be8528685cb639695766e7ee2f85fc9249877e7c6ade361e58a67ebd2f/merged major:0 minor:176 fsType:overlay blockSize:0} overlay_0-181:{mountpoint:/var/lib/containers/storage/overlay/10a9f0468f58094f2e336a0ebe845ff99f96984b72327e8ea26c8d0604ec1357/merged major:0 minor:181 fsType:overlay blockSize:0} overlay_0-186:{mountpoint:/var/lib/containers/storage/overlay/10cd1dec655fc8d39a088e3f025561a514ad7d05977c0302252652da6cb6bdb8/merged major:0 minor:186 fsType:overlay blockSize:0} overlay_0-191:{mountpoint:/var/lib/containers/storage/overlay/c7ba33491ecdb4dc163be3af5791393d73895139651b1fa3875f600cec3fbfe9/merged major:0 minor:191 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/0eb1db4874ca1b41c4b9ce3bfd84750e0b2d1cfaaf72d9d8cb201bc324fee730/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/61e054d962eebd1e7ad45907624c99363ee86c36bca735a91a27fddcdb3a917c/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/b002d1afe1f371e5ea596f67c596d3bebb55ad5fb0a0366f1a77f5df7f70f55c/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/882928ebfc46b5a2fdaa4eacaa10f4aee2b6f8d288a746e1b98aa77e99cd0579/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/3e95be5cbeff432b72762c8a3d0b0816a5b3161c76958840984a347a677e17ac/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/469084572b0e15350ad0337d981ce1fa6c6b570ac304cd1bef28b51df368a02c/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/35cf997ef49edeb4ac016586ea4187b3ed3cffa82082e4ff3b559c7b990e8f41/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/52263e74c4025787116903d5caee86d220a86386595d2229d2a5bbbfe633715b/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/c63832b7ab3fe635c4e3b0d717374073527c0f611f39cfcfa6470e8abf8e7859/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/3eb86ac127aab4540b57346c5aae9a20bdbc60b91c139b44a0e74167e3ce0d6a/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-255:{mountpoint:/var/lib/containers/storage/overlay/96ce3052fefa1396350dd4d4f44733b30f7160a6b7b822194f34eeba3a5fa2d5/merged major:0 minor:255 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/9e7f5413e7ec7bf64ec4e5a68c3fca326991c504943c24f8b9c4b0e87982c813/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/099cb1c2729b05817b87c8707136572155024d2206525eb374ebb77273ff7804/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/b576d79d65eab6c94c7d821ee4aed04a739ac1c55b601744068addafb3a13391/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/d3c11b27365c575b80a3b31d5d545e3bf04c16f26198dfb394ec7831301ab69e/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-288:{mountpoint:/var/lib/containers/storage/overlay/b39a894a204832c0dc5d5c471559e5aa3648e02dc0c50a8f24bb65e9d9cdd021/merged major:0 minor:288 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/b7351a4ff7c19ea7fc31191fb326049d979326e864eaa4210988b91bfa3ac7ab/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/98a76ea84f2d85f9fa9f8850e021501c90c81d3977f0a7e480942d800746b81f/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/2b75cc2c97aaaba242877e1b676922562c869d9a06bc16cbe92ca203c8025b7e/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/117abd2e7ad72beee03c682b89b5d086bdd25cb41eb85f56788c8f934249511e/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/ced22ab2c8b6d70c865220ea0d36febfc288671764761bb47a77f3f918b136d5/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-325:{mountpoint:/var/lib/containers/storage/overlay/788800b248c8d58d084081479a0ed6514430c8f8aa49880bb57f23e3f3c0ab75/merged major:0 minor:325 fsType:overlay blockSize:0} overlay_0-330:{mountpoint:/var/lib/containers/storage/overlay/39b66d304fba3d50fe7c34e1268c52b926bf69a2be47c8a37ad0dc4732ead9e3/merged major:0 minor:330 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/1653dc5b0a8269eebe55cc88848828a5fa6988c2b863cc3ae77c5283d6d48e0a/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/8fd8473611071c63d601f23b57aa363c38a884a18651f1e1016eb6e95deba26a/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/f60783d6a5a0785600b0559c36bb93fbcde5bf86b8ce7015f0454de36665eac7/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-365:{mountpoint:/var/lib/containers/storage/overlay/393b17d29f8bd4c04df5aaa144ece87a799affb134b795a69b9a930dc031ec4a/merged major:0 minor:365 fsType:overlay blockSize:0} overlay_0-367:{mountpoint:/var/lib/containers/storage/overlay/35bb726269aac87865bc5df277eae38f20930728c30ebf9a7bb932f76df71d0b/merged major:0 minor:367 fsType:overlay blockSize:0} overlay_0-369:{mountpoint:/var/lib/containers/storage/overlay/d4cbec1858ae6b8c9e218bc2bed6fb791c28ef489597c9163e0943985b8527bc/merged major:0 minor:369 fsType:overlay blockSize:0} overlay_0-371:{mountpoint:/var/lib/containers/storage/overlay/4fbdc40f507a120bb8b52423fc7aee32f8aef85bed19346e0bb2e039424d1a51/merged major:0 minor:371 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/f267bbb3eb69fe24bc80663235bc6483c8087eaef83e2f402b6869a86f759715/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-379:{mountpoint:/var/lib/containers/storage/overlay/3fb85e5eacc8f966f70541be44005e61d2278ba76aeb984ce7d93008c6187e69/merged major:0 minor:379 fsType:overlay blockSize:0} overlay_0-384:{mountpoint:/var/lib/containers/storage/overlay/0146d1602d7f20f10e286823806a07f8f5ab05b4c75887bc107d1e00ff5777ea/merged major:0 minor:384 fsType:overlay blockSize:0} overlay_0-45:{mountpoint:/var/lib/containers/storage/overlay/f51a9ac08c2c488685c6e5525c4ec05d6ae1b747524e16e9b185caf49f81cb58/merged major:0 minor:45 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/ed2c667020ed80087a186f299bc7af7e1d0a619fd7dc6f38aea3257db2dee96d/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/dce28ee3b0dcf37058a4797aa22a211e02ec2deec6520ca7d8f2c3d253798c70/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/4f982fc9b041c66d3b7c9dc6737f77fa64e164791611e06c65b8d3c1a21777fc/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/538488a3389f4a44d352525d44669fe2ffa33f272361bd1dabc26f6a094263d6/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/686444a4d48e4a8a52b303743b9067c1ba2d58071a5855e3776bf238d1c101f2/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/b92a36ad7a06645068ea0ad3156ca0472ee6c42302d9d6ae95a8f452a2cfe0a6/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/b095eb8762dbd84a947858de07193234bba4fc9f0d84833fb9afa0ec25ad72eb/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/4d29e3d941222aa9e7aed39c9ec87c52a0a15cefbb9adf3a4b009b72e87d4022/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/d91c6c0dd421907e0a5d664beca8edb0883272a85921499ee24afeefde62027d/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/dc77155414c250a172b44e9e3135d083f8c565fbd1302e76fcd0c16c855fb9d2/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.250128    5605 manager.go:217] Machine: {Timestamp:2026-11-03 16:11:54.249601667 +0000 UTC m=+0.074052929 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100658 MemoryCapacity:32812023808 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec289dff6cc424a9cf7ba9283ee178b4 SystemUUID:ec289dff-6cc4-24a9-cf7b-a9283ee178b4 BootID:ce1434b7-7734-4e5d-bfc9-d4797dba3384 Filesystems:[{Device:/var/lib/kubelet/pods/8ad82d3f-4298-4eb1-979b-91f94c007cef/volumes/kubernetes.io~projected/kube-api-access-hh6ld DeviceMajor:0 DeviceMinor:283 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/850880eb67311ce34cb01c81fdbbf517468eeaad7bf75d1fd085be358f4029b1/userdata/shm DeviceMajor:0 DeviceMinor:363 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f239d145-6b28-421b-a024-e86135e1fb4d/volumes/kubernetes.io~projected/kube-api-access-kw9c7 DeviceMajor:0 DeviceMinor:64 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/38701b60-1908-4568-81ed-0c5a91716678/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:60 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/d3a76124-dbf8-4fb3-bc58-52a21819b9d9/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:128 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f5a5565102010c4d5acf9f84f18c149ae4f0c16d9461cddbc8ff8e6bf14f7795/userdata/shm DeviceMajor:0 DeviceMinor:284 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/c6f6992c-48b9-4bb5-9b85-a37552b64c8b/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:357 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/dadfa10c-198c-4f93-869f-377048834934/volumes/kubernetes.io~projected/kube-api-access-pl446 DeviceMajor:0 DeviceMinor:56 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/8ad82d3f-4298-4eb1-979b-91f94c007cef/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:277 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/38701b60-1908-4568-81ed-0c5a91716678/volumes/kubernetes.io~projected/kube-api-access-sfl52 DeviceMajor:0 DeviceMinor:63 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/0599707f-1cdb-4e9f-99da-72703b028198/volumes/kubernetes.io~projected/kube-api-access-r7nln DeviceMajor:0 DeviceMinor:67 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-161 DeviceMajor:0 DeviceMinor:161 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-171 DeviceMajor:0 DeviceMinor:171 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/c9693f80-8f98-41c3-a72e-089d647d47a2/volumes/kubernetes.io~projected/kube-api-access-zwx5m DeviceMajor:0 DeviceMinor:240 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7f6d93c7656545212da5ac11e453f70ab88ee42e872c14e6c466cd8dbe78ad2c/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:298 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/99f565798e7ad8e99005e421011b93c887c108cb824c7635b668e019284a8056/userdata/shm DeviceMajor:0 DeviceMinor:346 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/cffd0b63-ba73-433f-886f-fcae960b41a1/volumes/kubernetes.io~projected/kube-api-access-sgqtp DeviceMajor:0 DeviceMinor:61 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eb0e217e600d3bdca1e147cbb5263225a859fa5159e34f9499c07b7567c1396c/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/74f17e70dee7783d3a22e38518b3bbc7b7defb865b7675ab1d7f610bae1c17f3/userdata/shm DeviceMajor:0 DeviceMinor:361 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/af1e5abda95474af2c5ae13d8bfbe3d225bd8c2911b774b657c401c97b3f0534/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-367 DeviceMajor:0 DeviceMinor:367 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1897a95d-8237-4a69-9ae8-c229490ba88c/volumes/kubernetes.io~projected/kube-api-access-k4pnj DeviceMajor:0 DeviceMinor:62 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/c14a9e56-182c-4abc-aa30-84b7958f775d/volumes/kubernetes.io~projected/kube-api-access-4brmn DeviceMajor:0 DeviceMinor:65 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-255 DeviceMajor:0 DeviceMinor:255 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:292 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:290 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/1c560131-5ae7-42ef-975b-23af6909fb89/volumes/kubernetes.io~projected/kube-api-access-c7l95 DeviceMajor:0 DeviceMinor:66 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/54d61260-b3aa-46f1-8cdd-d15102a208ce/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:296 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/76785a65-390d-4367-ac2b-6bf096c6b18d/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:344 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/d80b9f00-54a1-4809-9856-efb0ee40bd24/volumes/kubernetes.io~projected/kube-api-access-sjvvq DeviceMajor:0 DeviceMinor:55 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-176 DeviceMajor:0 DeviceMinor:176 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:291 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:295 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/1897a95d-8237-4a69-9ae8-c229490ba88c/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/bafe6e21-68bb-4c42-84bf-96e38b7c1e55/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:229 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-330 DeviceMajor:0 DeviceMinor:330 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-369 DeviceMajor:0 DeviceMinor:369 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/987d64b915ab5b9587adb18f4ec8af3d3d9179036bd21512ceebd2aa91a707fd/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d3a76124-dbf8-4fb3-bc58-52a21819b9d9/volumes/kubernetes.io~projected/kube-api-access-z56b7 DeviceMajor:0 DeviceMinor:202 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/ffeba40d-6416-4f6f-b645-d6e3338f7adb/volumes/kubernetes.io~projected/kube-api-access-jl42j DeviceMajor:0 DeviceMinor:203 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1897a95d-8237-4a69-9ae8-c229490ba88c/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-379 DeviceMajor:0 DeviceMinor:379 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406011904 Type:vfs Inodes:4005374 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6a15c0e001c60f152152a613b3c0365ac1f8c5032b7af7caadb111e177f8929c/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cffd0b63-ba73-433f-886f-fcae960b41a1/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:352 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2e2a0550fd2598e68d0ca1f4731c713f02134f627850a60044638e380f61fc2b/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-45 DeviceMajor:0 DeviceMinor:45 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-122 DeviceMajor:0 DeviceMinor:122 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:297 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/54997c0f21445fa127f7c31027a61b9a9c73aeb4b577450da384c4c885c5d6f7/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/92821c6fc58b809f6cd483310cd470578eb71531fc5f8dce6914f49fa9b015c1/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-166 DeviceMajor:0 DeviceMinor:166 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bafe6e21-68bb-4c42-84bf-96e38b7c1e55/volumes/kubernetes.io~projected/kube-api-access-mbshk DeviceMajor:0 DeviceMinor:230 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/a67334f9-d13e-452c-b266-e0e996dd7db3/volumes/kubernetes.io~projected/kube-api-access-zsxr9 DeviceMajor:0 DeviceMinor:358 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c9693f80-8f98-41c3-a72e-089d647d47a2/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:235 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:290 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-325 DeviceMajor:0 DeviceMinor:325 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4d67fc0bee521b1a0c4198f96b87ad9bc5ceea29046fbb617ad8e4076d91d853/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/38701b60-1908-4568-81ed-0c5a91716678/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:57 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~projected/kube-api-access-ddv4f DeviceMajor:0 DeviceMinor:299 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/8ad82d3f-4298-4eb1-979b-91f94c007cef/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:282 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f78dc792b42754170fd76c386076ebb557d32b1e906205156736d45231680942/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/29dad769d3401de44406bfe76091c3caa1c312098ec7c27d72c89c748eec01d8/userdata/shm DeviceMajor:0 DeviceMinor:89 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-181 DeviceMajor:0 DeviceMinor:181 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/54d61260-b3aa-46f1-8cdd-d15102a208ce/volumes/kubernetes.io~projected/kube-api-access-wzk2k DeviceMajor:0 DeviceMinor:204 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:294 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:294 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/76785a65-390d-4367-ac2b-6bf096c6b18d/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:340 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8f6f806a460f88a83e2b5fa8a4611360fa35f6a8872bcf20f755fe5e8e008d31/userdata/shm DeviceMajor:0 DeviceMinor:359 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:293 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:290 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/2cfc5077-0c47-48fd-a247-51156f01935c/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:59 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/839386ca47a23cf94d69213f6f6395de4fa713f3bfb6dd4b5192343ab1c0c969/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/648f1fa2-0c16-4abc-964b-200caa87def3/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:290 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-365 DeviceMajor:0 DeviceMinor:365 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-371 DeviceMajor:0 DeviceMinor:371 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ffeba40d-6416-4f6f-b645-d6e3338f7adb/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:200 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/var/lib/kubelet/pods/bafe6e21-68bb-4c42-84bf-96e38b7c1e55/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:225 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-288 DeviceMajor:0 DeviceMinor:288 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-384 DeviceMajor:0 DeviceMinor:384 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-191 DeviceMajor:0 DeviceMinor:191 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/76785a65-390d-4367-ac2b-6bf096c6b18d/volumes/kubernetes.io~projected/kube-api-access-4djl6 DeviceMajor:0 DeviceMinor:345 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/52fa4ae03cad2bfea8518e5e02e5c89a84500eaf6176c3682eabf55b08d4df93/userdata/shm DeviceMajor:0 DeviceMinor:83 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-186 DeviceMajor:0 DeviceMinor:186 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c9693f80-8f98-41c3-a72e-089d647d47a2/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:236 Capacity:31633424384 Type:vfs Inodes:4005374 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3019d8ce3631c97e275f3646298176c2f6355d9291eb32d4ada3fd0e87496b55/userdata/shm DeviceMajor:0 DeviceMinor:303 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/edbb68bb5cbcfaf8e36384356bc1f636bfc4922eaebe1e7d75ae041fd7ca91db/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f88d78bcdab59ab026d9e8b15d0f7dbd94b3b631627ecd7576c5dcbb205ab3f6/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005374 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:3019d8ce3631c97 MacAddress:c2:b7:10:c0:38:6c Speed:10000 Mtu:8901} {Name:74f17e70dee7783 MacAddress:ca:9c:45:3c:b8:1d Speed:10000 Mtu:8901} {Name:7f6d93c76565452 MacAddress:52:1f:70:3c:55:41 Speed:10000 Mtu:8901} {Name:839386ca47a23cf MacAddress:3a:4a:8f:20:b4:9e Speed:10000 Mtu:8901} {Name:850880eb67311ce MacAddress:aa:4d:a9:a9:86:19 Speed:10000 Mtu:8901} {Name:8f6f806a460f88a MacAddress:5a:94:c8:da:82:a2 Speed:10000 Mtu:8901} {Name:99f565798e7ad8e MacAddress:b2:30:35:24:4a:0b Speed:10000 Mtu:8901} {Name:af1e5abda95474a MacAddress:52:d6:23:3a:60:55 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:e6:94:04:7b Speed:0 Mtu:9001} {Name:br-int MacAddress:d6:a7:3f:a9:95:4f Speed:0 Mtu:8901} {Name:eb0e217e600d3bd MacAddress:fa:70:fb:a9:8d:06 Speed:10000 Mtu:8901} {Name:edbb68bb5cbcfaf MacAddress:9a:8b:0a:74:52:b5 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:e6:94:04:7b Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:42:96:df:e3:ca:c3 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:06:dd:7c:db:1b:f7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812023808 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.250617    5605 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.250668    5605 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.250959    5605 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.250981    5605 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-22.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251133    5605 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251150    5605 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251165    5605 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251184    5605 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251287    5605 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251477    5605 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251595    5605 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251639    5605 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251652    5605 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251662    5605 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251671    5605 kubelet.go:405] "Adding apiserver pod source"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.251680    5605 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.252923    5605 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.252939    5605 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 16:11:54.257300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.254086    5605 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 16:11:54.258172 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258155    5605 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 16:11:54.258204 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258172    5605 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 16:11:54.258285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258275    5605 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 03 16:11:54.258498 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258487    5605 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 16:11:54.258528 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258517    5605 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 16:11:54.258623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258614    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 16:11:54.258651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258628    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 16:11:54.258651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258635    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 16:11:54.258651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258641    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 16:11:54.258651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258647    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 16:11:54.258773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258653    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 16:11:54.258773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258660    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 16:11:54.258773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258665    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 16:11:54.258773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258673    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 16:11:54.258773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258679    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 16:11:54.258773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258688    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 16:11:54.258773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258697    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 16:11:54.258773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258721    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 16:11:54.258773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.258743    5605 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 16:11:54.262163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.262149    5605 server.go:1263] "Started kubelet"
Nov 03 16:11:54.262258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.262229    5605 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 16:11:54.263829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.262308    5605 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 16:11:54.263829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.262386    5605 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 16:11:54.263829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.262700    5605 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 16:11:54.263829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.263417    5605 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 16:11:54.262938 ip-10-0-142-22 systemd[1]: Started Kubernetes Kubelet.
Nov 03 16:11:54.265330 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.265315    5605 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 16:11:54.265455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.265391    5605 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 16:11:54.265455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.265405    5605 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 16:11:54.265608 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:54.265453    5605 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:11:54.265608 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.265511    5605 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 16:11:54.265608 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.265531    5605 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 16:11:54.265608 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.265570    5605 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 16:04:40 +0000 UTC" deadline="2028-03-31 06:04:37.885324875 +0000 UTC"
Nov 03 16:11:54.265608 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.265601    5605 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12325h52m43.619726373s"
Nov 03 16:11:54.266254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.266223    5605 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 16:11:54.266254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.266242    5605 factory.go:55] Registering systemd factory
Nov 03 16:11:54.266254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.266251    5605 factory.go:223] Registration of the systemd container factory successfully
Nov 03 16:11:54.267928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.266475    5605 factory.go:153] Registering CRI-O factory
Nov 03 16:11:54.267928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.266498    5605 factory.go:223] Registration of the crio container factory successfully
Nov 03 16:11:54.267928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.266511    5605 factory.go:103] Registering Raw factory
Nov 03 16:11:54.267928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.266522    5605 manager.go:1199] Started watching for new ooms in manager
Nov 03 16:11:54.267928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.266814    5605 manager.go:319] Starting recovery of all containers
Nov 03 16:11:54.268612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268583    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-main-tls" seLinuxMountContext=""
Nov 03 16:11:54.268679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268618    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Nov 03 16:11:54.268679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268628    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76785a65-390d-4367-ac2b-6bf096c6b18d" volumeName="kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-console-config" seLinuxMountContext=""
Nov 03 16:11:54.268679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268644    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" volumeName="kubernetes.io/projected/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-kube-api-access-mbshk" seLinuxMountContext=""
Nov 03 16:11:54.268679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268653    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0599707f-1cdb-4e9f-99da-72703b028198" volumeName="kubernetes.io/empty-dir/0599707f-1cdb-4e9f-99da-72703b028198-tmp-dir" seLinuxMountContext=""
Nov 03 16:11:54.268679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268660    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c560131-5ae7-42ef-975b-23af6909fb89" volumeName="kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-multus-daemon-config" seLinuxMountContext=""
Nov 03 16:11:54.268679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268667    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76785a65-390d-4367-ac2b-6bf096c6b18d" volumeName="kubernetes.io/projected/76785a65-390d-4367-ac2b-6bf096c6b18d-kube-api-access-4djl6" seLinuxMountContext=""
Nov 03 16:11:54.268679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268681    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a67334f9-d13e-452c-b266-e0e996dd7db3" volumeName="kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9" seLinuxMountContext=""
Nov 03 16:11:54.268679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268690    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" volumeName="kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-service-ca" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268710    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" volumeName="kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-oauth-serving-cert" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268718    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" volumeName="kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-trusted-ca-bundle" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268756    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268770    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ad82d3f-4298-4eb1-979b-91f94c007cef" volumeName="kubernetes.io/empty-dir/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-textfile" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268811    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" volumeName="kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-config" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268826    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d80b9f00-54a1-4809-9856-efb0ee40bd24" volumeName="kubernetes.io/projected/d80b9f00-54a1-4809-9856-efb0ee40bd24-kube-api-access-sjvvq" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268847    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ffeba40d-6416-4f6f-b645-d6e3338f7adb" volumeName="kubernetes.io/empty-dir/ffeba40d-6416-4f6f-b645-d6e3338f7adb-data-volume" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268861    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-main-db" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268883    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c9693f80-8f98-41c3-a72e-089d647d47a2" volumeName="kubernetes.io/configmap/c9693f80-8f98-41c3-a72e-089d647d47a2-metrics-client-ca" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268896    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dadfa10c-198c-4f93-869f-377048834934" volumeName="kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268917    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dadfa10c-198c-4f93-869f-377048834934" volumeName="kubernetes.io/projected/dadfa10c-198c-4f93-869f-377048834934-kube-api-access-pl446" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268930    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0599707f-1cdb-4e9f-99da-72703b028198" volumeName="kubernetes.io/projected/0599707f-1cdb-4e9f-99da-72703b028198-kube-api-access-r7nln" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268961    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-cluster-tls-config" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268974    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76785a65-390d-4367-ac2b-6bf096c6b18d" volumeName="kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-serving-cert" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.268996    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ad82d3f-4298-4eb1-979b-91f94c007cef" volumeName="kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-tls" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269007    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d80b9f00-54a1-4809-9856-efb0ee40bd24" volumeName="kubernetes.io/configmap/d80b9f00-54a1-4809-9856-efb0ee40bd24-serviceca" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269030    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c560131-5ae7-42ef-975b-23af6909fb89" volumeName="kubernetes.io/projected/1c560131-5ae7-42ef-975b-23af6909fb89-kube-api-access-c7l95" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269041    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ad82d3f-4298-4eb1-979b-91f94c007cef" volumeName="kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269062    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c14a9e56-182c-4abc-aa30-84b7958f775d" volumeName="kubernetes.io/configmap/c14a9e56-182c-4abc-aa30-84b7958f775d-iptables-alerter-script" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269089    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c9693f80-8f98-41c3-a72e-089d647d47a2" volumeName="kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 03 16:11:54.269089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269100    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ffeba40d-6416-4f6f-b645-d6e3338f7adb" volumeName="kubernetes.io/secret/ffeba40d-6416-4f6f-b645-d6e3338f7adb-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269111    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54d61260-b3aa-46f1-8cdd-d15102a208ce" volumeName="kubernetes.io/configmap/54d61260-b3aa-46f1-8cdd-d15102a208ce-config-volume" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269176    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269187    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c14a9e56-182c-4abc-aa30-84b7958f775d" volumeName="kubernetes.io/projected/c14a9e56-182c-4abc-aa30-84b7958f775d-kube-api-access-4brmn" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269206    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f239d145-6b28-421b-a024-e86135e1fb4d" volumeName="kubernetes.io/projected/f239d145-6b28-421b-a024-e86135e1fb4d-kube-api-access-kw9c7" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269217    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54d61260-b3aa-46f1-8cdd-d15102a208ce" volumeName="kubernetes.io/projected/54d61260-b3aa-46f1-8cdd-d15102a208ce-kube-api-access-wzk2k" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269253    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54d61260-b3aa-46f1-8cdd-d15102a208ce" volumeName="kubernetes.io/secret/54d61260-b3aa-46f1-8cdd-d15102a208ce-metrics-tls" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269297    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ad82d3f-4298-4eb1-979b-91f94c007cef" volumeName="kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-metrics-client-ca" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269315    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-metrics-client-ca" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269327    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-config-out" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269355    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269366    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ad82d3f-4298-4eb1-979b-91f94c007cef" volumeName="kubernetes.io/projected/8ad82d3f-4298-4eb1-979b-91f94c007cef-kube-api-access-hh6ld" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269395    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54d61260-b3aa-46f1-8cdd-d15102a208ce" volumeName="kubernetes.io/empty-dir/54d61260-b3aa-46f1-8cdd-d15102a208ce-tmp-dir" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269405    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-kube-api-access-ddv4f" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269424    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76785a65-390d-4367-ac2b-6bf096c6b18d" volumeName="kubernetes.io/empty-dir/76785a65-390d-4367-ac2b-6bf096c6b18d-tmp" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269437    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6f6992c-48b9-4bb5-9b85-a37552b64c8b" volumeName="kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269456    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dadfa10c-198c-4f93-869f-377048834934" volumeName="kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-binary-copy" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269467    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" volumeName="kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-oauth-config" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269493    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1897a95d-8237-4a69-9ae8-c229490ba88c" volumeName="kubernetes.io/secret/1897a95d-8237-4a69-9ae8-c229490ba88c-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269505    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="38701b60-1908-4568-81ed-0c5a91716678" volumeName="kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-etc-tuned" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269538    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76785a65-390d-4367-ac2b-6bf096c6b18d" volumeName="kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-service-ca" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269549    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76785a65-390d-4367-ac2b-6bf096c6b18d" volumeName="kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-oauth-serving-cert" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269568    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76785a65-390d-4367-ac2b-6bf096c6b18d" volumeName="kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-trusted-ca-bundle" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269579    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3a76124-dbf8-4fb3-bc58-52a21819b9d9" volumeName="kubernetes.io/projected/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-kube-api-access-z56b7" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269597    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" volumeName="kubernetes.io/empty-dir/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-tmp" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269616    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2cfc5077-0c47-48fd-a247-51156f01935c" volumeName="kubernetes.io/secret/2cfc5077-0c47-48fd-a247-51156f01935c-agent-certs" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269641    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ad82d3f-4298-4eb1-979b-91f94c007cef" volumeName="kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269675    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dadfa10c-198c-4f93-869f-377048834934" volumeName="kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269697    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1897a95d-8237-4a69-9ae8-c229490ba88c" volumeName="kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-config" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269708    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-tls-assets" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269718    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-config-volume" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269756    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cffd0b63-ba73-433f-886f-fcae960b41a1" volumeName="kubernetes.io/projected/cffd0b63-ba73-433f-886f-fcae960b41a1-kube-api-access-sgqtp" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269774    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ffeba40d-6416-4f6f-b645-d6e3338f7adb" volumeName="kubernetes.io/configmap/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269786    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1897a95d-8237-4a69-9ae8-c229490ba88c" volumeName="kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-env-overrides" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269824    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1897a95d-8237-4a69-9ae8-c229490ba88c" volumeName="kubernetes.io/projected/1897a95d-8237-4a69-9ae8-c229490ba88c-kube-api-access-k4pnj" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269852    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cffd0b63-ba73-433f-886f-fcae960b41a1" volumeName="kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269862    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ffeba40d-6416-4f6f-b645-d6e3338f7adb" volumeName="kubernetes.io/projected/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-api-access-jl42j" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269873    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76785a65-390d-4367-ac2b-6bf096c6b18d" volumeName="kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-oauth-config" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269908    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c9693f80-8f98-41c3-a72e-089d647d47a2" volumeName="kubernetes.io/projected/c9693f80-8f98-41c3-a72e-089d647d47a2-kube-api-access-zwx5m" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269919    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c9693f80-8f98-41c3-a72e-089d647d47a2" volumeName="kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-tls" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269960    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3a76124-dbf8-4fb3-bc58-52a21819b9d9" volumeName="kubernetes.io/secret/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-cert" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269971    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="38701b60-1908-4568-81ed-0c5a91716678" volumeName="kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-tmp" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.269991    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="648f1fa2-0c16-4abc-964b-200caa87def3" volumeName="kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-web-config" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.270016    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" volumeName="kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-serving-cert" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.270027    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1897a95d-8237-4a69-9ae8-c229490ba88c" volumeName="kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-script-lib" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.270038    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c560131-5ae7-42ef-975b-23af6909fb89" volumeName="kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-cni-binary-copy" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.270059    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2cfc5077-0c47-48fd-a247-51156f01935c" volumeName="kubernetes.io/configmap/2cfc5077-0c47-48fd-a247-51156f01935c-konnectivity-ca" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.270073    5605 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="38701b60-1908-4568-81ed-0c5a91716678" volumeName="kubernetes.io/projected/38701b60-1908-4568-81ed-0c5a91716678-kube-api-access-sfl52" seLinuxMountContext=""
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.270087    5605 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 16:11:54.270114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.270094    5605 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 16:11:54.285956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.285928    5605 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 16:11:54.296124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.296102    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:11:54.296593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.296556    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:11:54.300877 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.300852    5605 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 16:11:54.300969 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.300880    5605 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 16:11:54.300969 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.300937    5605 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 16:11:54.301137 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:54.300982    5605 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 03 16:11:54.323292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.323258    5605 generic.go:356] "Generic (PLEG): container finished" podID="648f1fa2-0c16-4abc-964b-200caa87def3" containerID="cda4f17ff7aa7d71b14326ec61f99c62b17d2aef5bca76098587796bf653cbd8" exitCode=0
Nov 03 16:11:54.331037 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.331004    5605 generic.go:356] "Generic (PLEG): container finished" podID="2677e9e089ae4c3ce22398c86c934d7d" containerID="0111f5c972adbd997ebaf61a26a04445e32592d1bd1fff16c4f40b031215ded2" exitCode=0
Nov 03 16:11:54.334754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.334713    5605 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="54710dc6b002c87f18bbde44e603e48c14afb9be933290aac8fd90f2b9a7012a" exitCode=0
Nov 03 16:11:54.334754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.334751    5605 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="3758f24798f703425eaf0fedaa6aa47fc3a5ddbc8a9492cd0f192436129d0bcc" exitCode=0
Nov 03 16:11:54.334754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.334758    5605 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="56d89659b0c3462af1d9ffa49bcb2b2f8169a1c68dd96f43c7b06aa3b22c6b9f" exitCode=0
Nov 03 16:11:54.334988 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.334766    5605 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="0d66c11959ae169f9b8dd0043755ffc4c9ce808663624cc847737e33304210c9" exitCode=0
Nov 03 16:11:54.334988 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.334772    5605 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="06d0183b5d8b230e9fbf0cfa183c72580e22f714b7a35602e0f9e7532c2c1168" exitCode=0
Nov 03 16:11:54.334988 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.334776    5605 generic.go:356] "Generic (PLEG): container finished" podID="dadfa10c-198c-4f93-869f-377048834934" containerID="7ca75a9c01edfbc25b850c51bd02c7f20879c88ce2f2f8f1dd867c36a70b9478" exitCode=0
Nov 03 16:11:54.342944 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.342925    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:11:54.343037 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.342953    5605 generic.go:356] "Generic (PLEG): container finished" podID="1c560131-5ae7-42ef-975b-23af6909fb89" containerID="b4f769c479f8a101ebab235dce7f85580ec680ba9c88955be5d8b1414879ab17" exitCode=1
Nov 03 16:11:54.349565 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.349544    5605 generic.go:356] "Generic (PLEG): container finished" podID="8ad82d3f-4298-4eb1-979b-91f94c007cef" containerID="5caaeae2e6d2f0075e2e8f4da6fd3215b50ea729b789746bfa1469c3e3bb514d" exitCode=0
Nov 03 16:11:54.365555 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:54.365527    5605 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:11:54.371465 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.371411    5605 manager.go:324] Recovery completed
Nov 03 16:11:54.392667 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.392632    5605 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:11:54.395787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.395761    5605 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:11:54.395920 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.395804    5605 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:11:54.395920 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.395821    5605 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:11:54.398564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398545    5605 cpu_manager.go:226] "Starting" policy="none"
Nov 03 16:11:54.398564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398560    5605 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 16:11:54.398564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398561    5605 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:11:54.398767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398577    5605 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 16:11:54.398767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398745    5605 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 03 16:11:54.398856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398760    5605 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 03 16:11:54.398856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398785    5605 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 03 16:11:54.398856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398793    5605 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 03 16:11:54.398856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398800    5605 policy_none.go:50] "Start"
Nov 03 16:11:54.398856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398812    5605 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 16:11:54.398856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.398827    5605 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 16:11:54.399506 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.399486    5605 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:11:54.399607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.399514    5605 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:11:54.399607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.399525    5605 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:11:54.400427 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.400409    5605 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 03 16:11:54.400427 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.400428    5605 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 03 16:11:54.400522 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.400434    5605 policy_none.go:44] "Start"
Nov 03 16:11:54.401635 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:54.401619    5605 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 03 16:11:54.404856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.404809    5605 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 16:11:54.411190 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.411176    5605 manager.go:341] "Starting Device Plugin manager"
Nov 03 16:11:54.411304 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:54.411204    5605 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 16:11:54.411304 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.411213    5605 server.go:87] "Starting device plugin registration server"
Nov 03 16:11:54.411514 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.411501    5605 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 16:11:54.411568 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.411513    5605 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 16:11:54.411621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.411604    5605 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 16:11:54.411779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.411750    5605 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 16:11:54.411848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.411788    5605 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 16:11:54.411848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.411794    5605 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 16:11:54.411941 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.411836    5605 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T16:11:54.411778226Z","UUID":"3a810c04-1a91-44c9-b929-19f5a61548eb","Handler":null,"Name":"","Endpoint":""}
Nov 03 16:11:54.413379 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:54.413355    5605 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 16:11:54.413471 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:54.413414    5605 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-22.us-west-2.compute.internal\" not found"
Nov 03 16:11:54.413882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.413868    5605 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 16:11:54.413933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.413889    5605 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 16:11:54.453140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.453105    5605 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 16:11:54.453285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.453188    5605 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 03 16:11:54.454502 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.454488    5605 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 16:11:54.499276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.499247    5605 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 16:11:54.512356 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.512329    5605 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 16:11:54.513358 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.513340    5605 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 03 16:11:54.513358 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.513370    5605 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 16:11:54.513501 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.513380    5605 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-22.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 03 16:11:54.513501 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.513423    5605 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.545678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.545653    5605 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 16:11:54.602692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.602631    5605 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"]
Nov 03 16:11:54.603381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.603319    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal" event={"ID":"2677e9e089ae4c3ce22398c86c934d7d","Type":"ContainerStarted","Data":"dc48f1009cc4cf1d6291e136430da1ffe1c43a34b96d8de362789f78c063542b"}
Nov 03 16:11:54.603381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.603381    5605 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.603560 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.603356    5605 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.603560 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.603386    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal" event={"ID":"2677e9e089ae4c3ce22398c86c934d7d","Type":"ContainerDied","Data":"0111f5c972adbd997ebaf61a26a04445e32592d1bd1fff16c4f40b031215ded2"}
Nov 03 16:11:54.603560 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.603522    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal" event={"ID":"2677e9e089ae4c3ce22398c86c934d7d","Type":"ContainerStarted","Data":"54997c0f21445fa127f7c31027a61b9a9c73aeb4b577450da384c4c885c5d6f7"}
Nov 03 16:11:54.603644 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.603597    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal" event={"ID":"608951668150694fd6003da288232820","Type":"ContainerStarted","Data":"e8330244d709ef9b9882a3140eebbd0ab3c4c38a8b166d504cb9e598af6aeb95"}
Nov 03 16:11:54.603644 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.603612    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal" event={"ID":"608951668150694fd6003da288232820","Type":"ContainerStarted","Data":"987d64b915ab5b9587adb18f4ec8af3d3d9179036bd21512ceebd2aa91a707fd"}
Nov 03 16:11:54.618505 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.618478    5605 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.618651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.618550    5605 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.665589 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.665525    5605 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 16:11:54.665589 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:54.665565    5605 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.666996 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.666984    5605 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 16:11:54.667055 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:54.667014    5605 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.755073 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.755049    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.755202 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.755079    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.755202 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.755095    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/608951668150694fd6003da288232820-config\") pod \"kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"608951668150694fd6003da288232820\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.855577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.855544    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.855577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.855579    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.855775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.855597    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/608951668150694fd6003da288232820-config\") pod \"kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"608951668150694fd6003da288232820\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.855775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.855644    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.855775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.855652    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2677e9e089ae4c3ce22398c86c934d7d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"2677e9e089ae4c3ce22398c86c934d7d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:54.855775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:54.855689    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/608951668150694fd6003da288232820-config\") pod \"kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal\" (UID: \"608951668150694fd6003da288232820\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:55.252564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.252529    5605 apiserver.go:52] "Watching apiserver"
Nov 03 16:11:55.351617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.351572    5605 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 16:11:55.352182 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.352160    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kbql5","openshift-ovn-kubernetes/ovnkube-node-qns9l","kube-system/global-pull-secret-syncer-tbpr9","openshift-cluster-node-tuning-operator/tuned-8k9vh","openshift-dns/dns-default-ffzs5","openshift-console/console-8b6c46946-f7klz","openshift-multus/multus-additional-cni-plugins-g2fnp","kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal","openshift-dns/node-resolver-j8mcw","kube-system/konnectivity-agent-pkn6d","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5","openshift-insights/insights-runtime-extractor-7ltc8","openshift-multus/network-metrics-daemon-k5rqd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal","openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5","openshift-multus/multus-qgfrq","openshift-network-diagnostics/network-check-target-lqvng","openshift-monitoring/alertmanager-main-0","openshift-image-registry/node-ca-gfrfv","openshift-console/console-7dd847f68b-wq7jc","openshift-monitoring/node-exporter-rbcxk","openshift-network-operator/iptables-alerter-864tf"]
Nov 03 16:11:55.366636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.366609    5605 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:55.366820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.366762    5605 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:55.367583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.367560    5605 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 16:11:55.404154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404100    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.404284 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404260    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.404347 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404330    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 16:11:55.404836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404800    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.404836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404828    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9gv4t\""
Nov 03 16:11:55.405203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404844    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-j7zlt\""
Nov 03 16:11:55.405203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404854    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zsc2l\""
Nov 03 16:11:55.405203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404849    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.405203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404872    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.405203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404897    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 16:11:55.405203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.404920    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.405203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405017    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 16:11:55.405203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405037    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.405203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405084    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.405203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405113    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405259    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405373    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405376    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405431    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405437    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405442    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405482    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tk7qx\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405520    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405530    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405534    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405525    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405565    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x72k5\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405573    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kkrgk\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405634    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4xv66\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405686    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405720    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405721    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xhvl5\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405779    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rqr4v\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405799    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-xzbs4\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405851    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405865    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 16:11:55.405865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405878    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405903    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-4vrm9\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.405970    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sxk2h\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406025    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406085    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406191    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406319    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406457    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4pfhl\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406506    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406536    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406562    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406586    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406630    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.406782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406775    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 16:11:55.407276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.406925    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-gswn6\""
Nov 03 16:11:55.442967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.442941    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.442967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.442967    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 16:11:55.445972 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.445959    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 16:11:55.446088 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.446073    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7zz2z\""
Nov 03 16:11:55.446219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.446115    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-x8k5l\""
Nov 03 16:11:55.446345 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.446286    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 16:11:55.446595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.446578    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 03 16:11:55.446912 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.446895    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 03 16:11:55.446912 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.446899    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 03 16:11:55.447061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.446923    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 03 16:11:55.447131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.447087    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m7glm\""
Nov 03 16:11:55.447388 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.447375    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 03 16:11:55.449083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.449057    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.450599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.450563    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jkpm9\""
Nov 03 16:11:55.450687 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.450612    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 03 16:11:55.450687 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.450614    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cbsvm\""
Nov 03 16:11:55.450687 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.450636    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 16:11:55.451007 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.450986    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 03 16:11:55.451007 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.450993    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.451158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451015    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 16:11:55.451158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.450997    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 16:11:55.451158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451016    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 03 16:11:55.451418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451404    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 16:11:55.451471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451425    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 03 16:11:55.451471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451409    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 03 16:11:55.451471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451445    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 03 16:11:55.451599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451482    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 03 16:11:55.451599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451505    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 03 16:11:55.451688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451607    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 03 16:11:55.451688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451620    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 03 16:11:55.451812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451782    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 03 16:11:55.451859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.451832    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 03 16:11:55.452519 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.452505    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 03 16:11:55.455419 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.455393    5605 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 16:11:55.455457 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:55.455444    5605 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:55.456804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.456792    5605 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 16:11:55.456841 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:11:55.456827    5605 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-22.us-west-2.compute.internal"
Nov 03 16:11:55.459077 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459061    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjvvq\" (UniqueName: \"kubernetes.io/projected/d80b9f00-54a1-4809-9856-efb0ee40bd24-kube-api-access-sjvvq\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:11:55.459114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459088    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-systemd-units\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.459151 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459113    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.459151 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459129    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-var-lib-kubelet\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.459242 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459155    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d61260-b3aa-46f1-8cdd-d15102a208ce-tmp-dir\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:11:55.459242 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459190    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-netns\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.459329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459270    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d61260-b3aa-46f1-8cdd-d15102a208ce-tmp-dir\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:11:55.459329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459284    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-sys\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.459329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459311    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-config\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.459473 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459343    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-socket-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.459473 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459374    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl446\" (UniqueName: \"kubernetes.io/projected/dadfa10c-198c-4f93-869f-377048834934-kube-api-access-pl446\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.459473 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459419    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.459609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459526    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-oauth-config\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.459609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459547    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.459609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459565    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-netd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.459609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459589    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z56b7\" (UniqueName: \"kubernetes.io/projected/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-kube-api-access-z56b7\") pod \"ingress-canary-kbql5\" (UID: \"d3a76124-dbf8-4fb3-bc58-52a21819b9d9\") " pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:11:55.459822 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459614    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.459822 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459677    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-lib-modules\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.459822 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459713    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-oauth-serving-cert\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.459822 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459753    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.459822 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459812    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-cnibin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459830    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459875    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-oauth-config\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459882    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-multus-certs\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459875    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459913    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d80b9f00-54a1-4809-9856-efb0ee40bd24-host\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459943    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.459981    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-netns\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460002    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-oauth-serving-cert\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460015    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460024    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-serving-cert\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460054    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-trusted-ca-bundle\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460089    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-config\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460103    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54d61260-b3aa-46f1-8cdd-d15102a208ce-metrics-tls\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460126    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d80b9f00-54a1-4809-9856-efb0ee40bd24-serviceca\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:11:55.460138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460142    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-log-socket\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460166    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ffeba40d-6416-4f6f-b645-d6e3338f7adb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460172    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460189    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-etc-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460273    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-serving-cert\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460298    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54d61260-b3aa-46f1-8cdd-d15102a208ce-metrics-tls\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460295    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-env-overrides\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460363    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d80b9f00-54a1-4809-9856-efb0ee40bd24-serviceca\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460355    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-systemd\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460485    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ffeba40d-6416-4f6f-b645-d6e3338f7adb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460497    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-env-overrides\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460489    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-kubelet\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460529    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460569    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460637    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460668    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0599707f-1cdb-4e9f-99da-72703b028198-hosts-file\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:11:55.460689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460690    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-run\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460708    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2cfc5077-0c47-48fd-a247-51156f01935c-konnectivity-ca\") pod \"konnectivity-agent-pkn6d\" (UID: \"2cfc5077-0c47-48fd-a247-51156f01935c\") " pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460743    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cffd0b63-ba73-433f-886f-fcae960b41a1-metrics-certs\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460741    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-systemd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460799    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-var-lib-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460856    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-cert\") pod \"ingress-canary-kbql5\" (UID: \"d3a76124-dbf8-4fb3-bc58-52a21819b9d9\") " pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460880    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460894    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-trusted-ca-bundle\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460932    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwx5m\" (UniqueName: \"kubernetes.io/projected/c9693f80-8f98-41c3-a72e-089d647d47a2-kube-api-access-zwx5m\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460944    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2cfc5077-0c47-48fd-a247-51156f01935c-konnectivity-ca\") pod \"konnectivity-agent-pkn6d\" (UID: \"2cfc5077-0c47-48fd-a247-51156f01935c\") " pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460967    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-tmp\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.460992    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2cfc5077-0c47-48fd-a247-51156f01935c-agent-certs\") pod \"konnectivity-agent-pkn6d\" (UID: \"2cfc5077-0c47-48fd-a247-51156f01935c\") " pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461018    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-sys\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461026    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-tmp\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461043    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-host\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461050    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461082    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-cert\") pod \"ingress-canary-kbql5\" (UID: \"d3a76124-dbf8-4fb3-bc58-52a21819b9d9\") " pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461100    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c14a9e56-182c-4abc-aa30-84b7958f775d-host-slash\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461147    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-registration-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461177    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461194    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-sys-fs\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461224    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-cni-binary-copy\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461261    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461303    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461329    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2cfc5077-0c47-48fd-a247-51156f01935c-agent-certs\") pod \"konnectivity-agent-pkn6d\" (UID: \"2cfc5077-0c47-48fd-a247-51156f01935c\") " pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461351    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9693f80-8f98-41c3-a72e-089d647d47a2-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461406    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-service-ca\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461417    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-cni-binary-copy\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.461438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461432    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76785a65-390d-4367-ac2b-6bf096c6b18d-tmp\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461470    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-multus\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461483    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9693f80-8f98-41c3-a72e-089d647d47a2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461495    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76785a65-390d-4367-ac2b-6bf096c6b18d-tmp\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461508    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-metrics-client-ca\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461530    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-modprobe-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461561    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-service-ca\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461585    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461590    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9693f80-8f98-41c3-a72e-089d647d47a2-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461614    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-k8s-cni-cncf-io\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461691    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-multus-daemon-config\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461703    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ad82d3f-4298-4eb1-979b-91f94c007cef-metrics-client-ca\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461708    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-root\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461715    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461851    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-textfile\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461872    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c560131-5ae7-42ef-975b-23af6909fb89-multus-daemon-config\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461881    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysconfig\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461919    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-textfile\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461933    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461964    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461982    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-hostroot\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.461996    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-ovn\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462011    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-config\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462026    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-kubelet-config\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462058    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-tmp\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462141    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462169    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw9c7\" (UniqueName: \"kubernetes.io/projected/f239d145-6b28-421b-a024-e86135e1fb4d-kube-api-access-kw9c7\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462172    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-tmp\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462188    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-config\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462213    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54d61260-b3aa-46f1-8cdd-d15102a208ce-config-volume\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462250    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-system-cni-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462289    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462288    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ffeba40d-6416-4f6f-b645-d6e3338f7adb-crio-socket\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462322    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-kubernetes\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462346    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-conf\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462368    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfl52\" (UniqueName: \"kubernetes.io/projected/38701b60-1908-4568-81ed-0c5a91716678-kube-api-access-sfl52\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462385    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-service-ca\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462396    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54d61260-b3aa-46f1-8cdd-d15102a208ce-config-volume\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462400    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c14a9e56-182c-4abc-aa30-84b7958f775d-iptables-alerter-script\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462461    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462481    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-bin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462496    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462512    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-web-config\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462528    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462539    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-service-ca\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462550    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c14a9e56-182c-4abc-aa30-84b7958f775d-iptables-alerter-script\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462564    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4brmn\" (UniqueName: \"kubernetes.io/projected/c14a9e56-182c-4abc-aa30-84b7958f775d-kube-api-access-4brmn\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:11:55.462688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462684    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pnj\" (UniqueName: \"kubernetes.io/projected/1897a95d-8237-4a69-9ae8-c229490ba88c-kube-api-access-k4pnj\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462759    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462765    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462784    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462818    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzk2k\" (UniqueName: \"kubernetes.io/projected/54d61260-b3aa-46f1-8cdd-d15102a208ce-kube-api-access-wzk2k\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462823    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-web-config\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462849    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462877    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-socket-dir-parent\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462890    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-original-pull-secret\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462904    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbshk\" (UniqueName: \"kubernetes.io/projected/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-kube-api-access-mbshk\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462932    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgqtp\" (UniqueName: \"kubernetes.io/projected/cffd0b63-ba73-433f-886f-fcae960b41a1-kube-api-access-sgqtp\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462967    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.462957    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463003    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463029    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7l95\" (UniqueName: \"kubernetes.io/projected/1c560131-5ae7-42ef-975b-23af6909fb89-kube-api-access-c7l95\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463087    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463101    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-kubelet\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463167    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463204    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffeba40d-6416-4f6f-b645-d6e3338f7adb-data-volume\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463242    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl42j\" (UniqueName: \"kubernetes.io/projected/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-api-access-jl42j\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463305    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ffeba40d-6416-4f6f-b645-d6e3338f7adb-data-volume\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463336    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4djl6\" (UniqueName: \"kubernetes.io/projected/76785a65-390d-4367-ac2b-6bf096c6b18d-kube-api-access-4djl6\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463368    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-binary-copy\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463385    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463415    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1897a95d-8237-4a69-9ae8-c229490ba88c-ovn-node-metrics-cert\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463435    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-oauth-serving-cert\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463482    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-config-volume\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463506    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-etc-kubernetes\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463532    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7nln\" (UniqueName: \"kubernetes.io/projected/0599707f-1cdb-4e9f-99da-72703b028198-kube-api-access-r7nln\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463555    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-slash\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463597    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-script-lib\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463643    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-oauth-serving-cert\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463692    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/648f1fa2-0c16-4abc-964b-200caa87def3-config-volume\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463699    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-device-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463748    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1897a95d-8237-4a69-9ae8-c229490ba88c-ovn-node-metrics-cert\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463758    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-conf-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463808    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-oauth-config\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463764    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dadfa10c-198c-4f93-869f-377048834934-cni-binary-copy\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463856    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1897a95d-8237-4a69-9ae8-c229490ba88c-ovnkube-script-lib\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463852    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-cnibin\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463895    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-config-out\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463932    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463952    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-oauth-config\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463954    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.463960    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/648f1fa2-0c16-4abc-964b-200caa87def3-config-out\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464002    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddv4f\" (UniqueName: \"kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-kube-api-access-ddv4f\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464363    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-node-log\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464419    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-tls\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464468    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-wtmp\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.464492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464506    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-console-config\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464547    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-etc-tuned\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464582    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-serving-cert\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464603    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464609    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-trusted-ca-bundle\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464646    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-system-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464680    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/648f1fa2-0c16-4abc-964b-200caa87def3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.464956    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-tls\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.465014    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0599707f-1cdb-4e9f-99da-72703b028198-tmp-dir\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.465053    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh6ld\" (UniqueName: \"kubernetes.io/projected/8ad82d3f-4298-4eb1-979b-91f94c007cef-kube-api-access-hh6ld\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.465089    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.465117    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-console-config\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.465162    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-bin\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.465198    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0599707f-1cdb-4e9f-99da-72703b028198-tmp-dir\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.465261    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-dbus\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.465408    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38701b60-1908-4568-81ed-0c5a91716678-etc-tuned\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.465446    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-trusted-ca-bundle\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.466670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.465487    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-serving-cert\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.516568 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.516474    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjvvq\" (UniqueName: \"kubernetes.io/projected/d80b9f00-54a1-4809-9856-efb0ee40bd24-kube-api-access-sjvvq\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:11:55.516882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.516860    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl446\" (UniqueName: \"kubernetes.io/projected/dadfa10c-198c-4f93-869f-377048834934-kube-api-access-pl446\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.516966 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.516937    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw9c7\" (UniqueName: \"kubernetes.io/projected/f239d145-6b28-421b-a024-e86135e1fb4d-kube-api-access-kw9c7\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.517043 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.517012    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z56b7\" (UniqueName: \"kubernetes.io/projected/d3a76124-dbf8-4fb3-bc58-52a21819b9d9-kube-api-access-z56b7\") pod \"ingress-canary-kbql5\" (UID: \"d3a76124-dbf8-4fb3-bc58-52a21819b9d9\") " pod="openshift-ingress-canary/ingress-canary-kbql5"
Nov 03 16:11:55.517219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.517197    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwx5m\" (UniqueName: \"kubernetes.io/projected/c9693f80-8f98-41c3-a72e-089d647d47a2-kube-api-access-zwx5m\") pod \"prometheus-operator-9cd89b74c-kdrd5\" (UID: \"c9693f80-8f98-41c3-a72e-089d647d47a2\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kdrd5"
Nov 03 16:11:55.518919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.518897    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsxr9\" (UniqueName: \"kubernetes.io/projected/a67334f9-d13e-452c-b266-e0e996dd7db3-kube-api-access-zsxr9\") pod \"network-check-target-lqvng\" (UID: \"a67334f9-d13e-452c-b266-e0e996dd7db3\") " pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:11:55.519019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.518921    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfl52\" (UniqueName: \"kubernetes.io/projected/38701b60-1908-4568-81ed-0c5a91716678-kube-api-access-sfl52\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.519019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.518982    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzk2k\" (UniqueName: \"kubernetes.io/projected/54d61260-b3aa-46f1-8cdd-d15102a208ce-kube-api-access-wzk2k\") pod \"dns-default-ffzs5\" (UID: \"54d61260-b3aa-46f1-8cdd-d15102a208ce\") " pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:11:55.519249 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.519226    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddv4f\" (UniqueName: \"kubernetes.io/projected/648f1fa2-0c16-4abc-964b-200caa87def3-kube-api-access-ddv4f\") pod \"alertmanager-main-0\" (UID: \"648f1fa2-0c16-4abc-964b-200caa87def3\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 16:11:55.519468 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.519449    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4djl6\" (UniqueName: \"kubernetes.io/projected/76785a65-390d-4367-ac2b-6bf096c6b18d-kube-api-access-4djl6\") pod \"console-8b6c46946-f7klz\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") " pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:11:55.519529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.519467    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbshk\" (UniqueName: \"kubernetes.io/projected/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-kube-api-access-mbshk\") pod \"console-7dd847f68b-wq7jc\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") " pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:11:55.519580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.519528    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7l95\" (UniqueName: \"kubernetes.io/projected/1c560131-5ae7-42ef-975b-23af6909fb89-kube-api-access-c7l95\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.519657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.519642    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4pnj\" (UniqueName: \"kubernetes.io/projected/1897a95d-8237-4a69-9ae8-c229490ba88c-kube-api-access-k4pnj\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.519707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.519691    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4brmn\" (UniqueName: \"kubernetes.io/projected/c14a9e56-182c-4abc-aa30-84b7958f775d-kube-api-access-4brmn\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:11:55.520589 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.520567    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl42j\" (UniqueName: \"kubernetes.io/projected/ffeba40d-6416-4f6f-b645-d6e3338f7adb-kube-api-access-jl42j\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.520873 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.520856    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh6ld\" (UniqueName: \"kubernetes.io/projected/8ad82d3f-4298-4eb1-979b-91f94c007cef-kube-api-access-hh6ld\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.520960 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.520945    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7nln\" (UniqueName: \"kubernetes.io/projected/0599707f-1cdb-4e9f-99da-72703b028198-kube-api-access-r7nln\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:11:55.521458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.521442    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgqtp\" (UniqueName: \"kubernetes.io/projected/cffd0b63-ba73-433f-886f-fcae960b41a1-kube-api-access-sgqtp\") pod \"network-metrics-daemon-k5rqd\" (UID: \"cffd0b63-ba73-433f-886f-fcae960b41a1\") " pod="openshift-multus/network-metrics-daemon-k5rqd"
Nov 03 16:11:55.566637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566607    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-multus\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.566769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566654    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-modprobe-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.566769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566675    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-k8s-cni-cncf-io\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.566769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566698    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-root\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.566769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566718    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysconfig\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.566769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566755    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-k8s-cni-cncf-io\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566740    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-multus\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566784    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-root\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566795    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-hostroot\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566802    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysconfig\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566760    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-hostroot\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566845    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-ovn\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566863    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-kubelet-config\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566887    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566818    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-modprobe-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566907    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-ovn\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566913    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-system-cni-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566936    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ffeba40d-6416-4f6f-b645-d6e3338f7adb-crio-socket\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566950    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-kubernetes\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566956    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-kubelet-config\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:55.566980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566964    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-conf\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.566950    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567013    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-bin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567019    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ffeba40d-6416-4f6f-b645-d6e3338f7adb-crio-socket\") pod \"insights-runtime-extractor-7ltc8\" (UID: \"ffeba40d-6416-4f6f-b645-d6e3338f7adb\") " pod="openshift-insights/insights-runtime-extractor-7ltc8"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567028    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-system-cni-dir\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567017    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-kubernetes\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567051    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-cni-bin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567054    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-socket-dir-parent\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567061    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-conf\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567083    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567106    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-kubelet\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567122    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-socket-dir-parent\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567136    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567160    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-kubelet\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567165    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567173    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-etc-kubernetes\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567192    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567195    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-slash\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567223    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-device-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567236    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-slash\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567243    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-etc-kubernetes\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567245    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-conf-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567276    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-device-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567278    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-cnibin\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567284    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-multus-conf-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567296    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dadfa10c-198c-4f93-869f-377048834934-cnibin\") pod \"multus-additional-cni-plugins-g2fnp\" (UID: \"dadfa10c-198c-4f93-869f-377048834934\") " pod="openshift-multus/multus-additional-cni-plugins-g2fnp"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567302    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-node-log\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567321    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-wtmp\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567339    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-system-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567355    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567386    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-node-exporter-wtmp\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567399    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-node-log\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567404    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-bin\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567413    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-system-cni-dir\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567421    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-bin\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567424    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-dbus\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567435    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-sysctl-d\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567445    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-systemd-units\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567455    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6f6992c-48b9-4bb5-9b85-a37552b64c8b-dbus\") pod \"global-pull-secret-syncer-tbpr9\" (UID: \"c6f6992c-48b9-4bb5-9b85-a37552b64c8b\") " pod="kube-system/global-pull-secret-syncer-tbpr9"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567462    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567470    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-systemd-units\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567480    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-var-lib-kubelet\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567485    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567496    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-netns\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567511    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-sys\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567525    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-socket-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567542    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567545    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-run-netns\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567555    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-netd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567576    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-sys\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567581    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-host-cni-netd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567581    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-lib-modules\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567601    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-var-lib-kubelet\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567601    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-socket-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567616    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-cnibin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567628    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567641    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-multus-certs\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.567612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567647    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-lib-modules\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567664    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d80b9f00-54a1-4809-9856-efb0ee40bd24-host\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567687    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-netns\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567693    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-multus-certs\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567704    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-log-socket\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567705    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d80b9f00-54a1-4809-9856-efb0ee40bd24-host\") pod \"node-ca-gfrfv\" (UID: \"d80b9f00-54a1-4809-9856-efb0ee40bd24\") " pod="openshift-image-registry/node-ca-gfrfv"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567719    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-run-netns\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567750    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-cnibin\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567760    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-log-socket\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567784    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-etc-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567800    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-systemd\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567830    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-etc-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567815    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-kubelet\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567847    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-etc-systemd\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567866    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0599707f-1cdb-4e9f-99da-72703b028198-hosts-file\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567874    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c560131-5ae7-42ef-975b-23af6909fb89-host-var-lib-kubelet\") pod \"multus-qgfrq\" (UID: \"1c560131-5ae7-42ef-975b-23af6909fb89\") " pod="openshift-multus/multus-qgfrq"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567882    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-run\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567900    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-systemd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567921    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-var-lib-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567939    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-sys\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567954    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-host\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567954    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-systemd\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567958    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-run\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567971    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c14a9e56-182c-4abc-aa30-84b7958f775d-host-slash\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567979    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0599707f-1cdb-4e9f-99da-72703b028198-hosts-file\") pod \"node-resolver-j8mcw\" (UID: \"0599707f-1cdb-4e9f-99da-72703b028198\") " pod="openshift-dns/node-resolver-j8mcw"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.567994    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad82d3f-4298-4eb1-979b-91f94c007cef-sys\") pod \"node-exporter-rbcxk\" (UID: \"8ad82d3f-4298-4eb1-979b-91f94c007cef\") " pod="openshift-monitoring/node-exporter-rbcxk"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568000    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-registration-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568008    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c14a9e56-182c-4abc-aa30-84b7958f775d-host-slash\") pod \"iptables-alerter-864tf\" (UID: \"c14a9e56-182c-4abc-aa30-84b7958f775d\") " pod="openshift-network-operator/iptables-alerter-864tf"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568003    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-var-lib-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568018    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38701b60-1908-4568-81ed-0c5a91716678-host\") pod \"tuned-8k9vh\" (UID: \"38701b60-1908-4568-81ed-0c5a91716678\") " pod="openshift-cluster-node-tuning-operator/tuned-8k9vh"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568023    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568036    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-registration-dir\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568053    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-etc-selinux\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568055    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-sys-fs\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568078    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568097    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f239d145-6b28-421b-a024-e86135e1fb4d-sys-fs\") pod \"aws-ebs-csi-driver-node-ckzp5\" (UID: \"f239d145-6b28-421b-a024-e86135e1fb4d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ckzp5"
Nov 03 16:11:55.569245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:55.568122    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1897a95d-8237-4a69-9ae8-c229490ba88c-run-openvswitch\") pod \"ovnkube-node-qns9l\" (UID: \"1897a95d-8237-4a69-9ae8-c229490ba88c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:56.765888 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:56.765859    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ffzs5"
Nov 03 16:11:56.953623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:56.953590    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:58.812272 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:58.812243    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:58.812632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:58.812480    5605 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 16:11:59.703787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:59.703759    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qns9l"
Nov 03 16:11:59.996489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:11:59.996411    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:12:03.389903 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:12:03.389864    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:12:05.202328 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:05.202277    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:12:20.379436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.379363    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7dd847f68b-wq7jc" podUID="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" containerName="console" containerID="cri-o://289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d" gracePeriod=15
Nov 03 16:12:20.629846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.629792    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:12:20.727222 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727192    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-oauth-serving-cert\") pod \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") "
Nov 03 16:12:20.727370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727257    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-oauth-config\") pod \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") "
Nov 03 16:12:20.727370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727289    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-kube-api-access-mbshk\" (UniqueName: \"kubernetes.io/projected/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-kube-api-access-mbshk\") pod \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") "
Nov 03 16:12:20.727370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727305    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-service-ca\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-service-ca\") pod \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") "
Nov 03 16:12:20.727370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727322    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-serving-cert\") pod \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") "
Nov 03 16:12:20.727370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727347    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-trusted-ca-bundle\") pod \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") "
Nov 03 16:12:20.727370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727363    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-config\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-config\") pod \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") "
Nov 03 16:12:20.727658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727387    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-tmp\" (UniqueName: \"kubernetes.io/empty-dir/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-tmp\") pod \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\" (UID: \"bafe6e21-68bb-4c42-84bf-96e38b7c1e55\") "
Nov 03 16:12:20.727658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727590    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-oauth-serving-cert" pod "bafe6e21-68bb-4c42-84bf-96e38b7c1e55" (UID: "bafe6e21-68bb-4c42-84bf-96e38b7c1e55"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:12:20.727796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727777    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-tmp" pod "bafe6e21-68bb-4c42-84bf-96e38b7c1e55" (UID: "bafe6e21-68bb-4c42-84bf-96e38b7c1e55"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:12:20.727896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727874    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-trusted-ca-bundle" pod "bafe6e21-68bb-4c42-84bf-96e38b7c1e55" (UID: "bafe6e21-68bb-4c42-84bf-96e38b7c1e55"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:12:20.727949 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727847    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-service-ca" pod "bafe6e21-68bb-4c42-84bf-96e38b7c1e55" (UID: "bafe6e21-68bb-4c42-84bf-96e38b7c1e55"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:12:20.728015 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.727991    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-config" pod "bafe6e21-68bb-4c42-84bf-96e38b7c1e55" (UID: "bafe6e21-68bb-4c42-84bf-96e38b7c1e55"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:12:20.729921 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.729894    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-serving-cert" pod "bafe6e21-68bb-4c42-84bf-96e38b7c1e55" (UID: "bafe6e21-68bb-4c42-84bf-96e38b7c1e55"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:12:20.730023 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.729980    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-oauth-config" pod "bafe6e21-68bb-4c42-84bf-96e38b7c1e55" (UID: "bafe6e21-68bb-4c42-84bf-96e38b7c1e55"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:12:20.730023 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.729977    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-kube-api-access-mbshk" pod "bafe6e21-68bb-4c42-84bf-96e38b7c1e55" (UID: "bafe6e21-68bb-4c42-84bf-96e38b7c1e55"). InnerVolumeSpecName "kube-api-access-mbshk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:12:20.828384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.828341    5605 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-oauth-config\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:12:20.828384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.828375    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbshk\" (UniqueName: \"kubernetes.io/projected/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-kube-api-access-mbshk\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:12:20.828384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.828387    5605 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:12:20.828384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.828397    5605 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-serving-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:12:20.828653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.828407    5605 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-trusted-ca-bundle\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:12:20.828653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.828416    5605 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-console-config\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:12:20.828653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.828425    5605 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-tmp\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:12:20.828653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:20.828434    5605 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bafe6e21-68bb-4c42-84bf-96e38b7c1e55-oauth-serving-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:12:21.450623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:21.450588    5605 generic.go:356] "Generic (PLEG): container finished" podID="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" containerID="289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d" exitCode=0
Nov 03 16:12:21.451064 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:21.450633    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dd847f68b-wq7jc"
Nov 03 16:12:21.451064 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:21.450673    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dd847f68b-wq7jc" event={"ID":"bafe6e21-68bb-4c42-84bf-96e38b7c1e55","Type":"ContainerDied","Data":"289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d"}
Nov 03 16:12:21.451064 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:21.450736    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dd847f68b-wq7jc" event={"ID":"bafe6e21-68bb-4c42-84bf-96e38b7c1e55","Type":"ContainerDied","Data":"eb0e217e600d3bdca1e147cbb5263225a859fa5159e34f9499c07b7567c1396c"}
Nov 03 16:12:21.451064 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:21.450755    5605 scope.go:122] "RemoveContainer" containerID="289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d"
Nov 03 16:12:21.460919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:21.460868    5605 scope.go:122] "RemoveContainer" containerID="289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d"
Nov 03 16:12:21.461554 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:12:21.461532    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d\": container with ID starting with 289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d not found: ID does not exist" containerID="289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d"
Nov 03 16:12:21.461616 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:21.461564    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d"} err="failed to get container status \"289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d\": rpc error: code = NotFound desc = could not find container \"289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d\": container with ID starting with 289bce02c813a7d9f45ededd89c16b505373d964477d6b3cced9017e778db21d not found: ID does not exist"
Nov 03 16:12:21.608131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:21.608096    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dd847f68b-wq7jc"]
Nov 03 16:12:21.654796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:21.654764    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dd847f68b-wq7jc"]
Nov 03 16:12:22.305976 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:22.305936    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" path="/var/lib/kubelet/pods/bafe6e21-68bb-4c42-84bf-96e38b7c1e55/volumes"
Nov 03 16:12:22.366972 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:22.366944    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pkn6d"
Nov 03 16:12:25.205689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:25.205655    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lqvng"
Nov 03 16:12:27.107236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.107199    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"]
Nov 03 16:12:27.107617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.107448    5605 cpu_manager.go:414] "containerMap: removing container" podUID="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" containerName="console"
Nov 03 16:12:27.107617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.107458    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" containerName="console"
Nov 03 16:12:27.107617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.107511    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="bafe6e21-68bb-4c42-84bf-96e38b7c1e55" containerName="console"
Nov 03 16:12:27.119797 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.119772    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.171079 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.171050    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 03 16:12:27.171830 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.171806    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 03 16:12:27.176113 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.176093    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-federate-client-tls\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.176179 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.176123    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-secret-telemeter-client\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.176179 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.176145    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.176179 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.176163    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-metrics-client-ca\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.176299 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.176237    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-serving-certs-ca-bundle\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.176340 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.176317    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-telemeter-client-tls\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.176377 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.176367    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.176473 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.176453    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfxkn\" (UniqueName: \"kubernetes.io/projected/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-kube-api-access-rfxkn\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.216902 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.216874    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 03 16:12:27.216902 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.216875    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 03 16:12:27.217228 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.217196    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 03 16:12:27.217298 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.217239    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wfh2s\""
Nov 03 16:12:27.264330 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.264291    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 03 16:12:27.265686 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.265665    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"]
Nov 03 16:12:27.277579 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.277556    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.277685 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.277599    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxkn\" (UniqueName: \"kubernetes.io/projected/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-kube-api-access-rfxkn\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.277685 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.277638    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-federate-client-tls\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.277685 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.277664    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-secret-telemeter-client\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.277864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.277688    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.277864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.277739    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-metrics-client-ca\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.277864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.277765    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-serving-certs-ca-bundle\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.278002 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.277888    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-telemeter-client-tls\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.278125 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.278105    5605 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 16:12:27.278528 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.278496    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-metrics-client-ca\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.278845 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.278802    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.278845 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.278808    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-serving-certs-ca-bundle\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.280472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.280451    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-telemeter-client-tls\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.280699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.280681    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-secret-telemeter-client\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.280963 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.280942    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.280963 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.280956    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-federate-client-tls\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.333398 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.333352    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfxkn\" (UniqueName: \"kubernetes.io/projected/659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd-kube-api-access-rfxkn\") pod \"telemeter-client-84f46ccbf-fkr8b\" (UID: \"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd\") " pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.470620 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.470532    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"
Nov 03 16:12:27.966817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.966692    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-84f46ccbf-fkr8b"]
Nov 03 16:12:27.971142 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:12:27.971113    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod659f80e7_2a5c_48b0_a2c8_e26fd10a4ffd.slice/crio-f47f23ab56fb40b496566e8c6286d6ed82f572309928e01110047b513fc3eb73 WatchSource:0}: Error finding container f47f23ab56fb40b496566e8c6286d6ed82f572309928e01110047b513fc3eb73: Status 404 returned error can't find the container with id f47f23ab56fb40b496566e8c6286d6ed82f572309928e01110047b513fc3eb73
Nov 03 16:12:27.973226 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:27.973209    5605 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 16:12:28.478305 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:28.478231    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b" event={"ID":"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd","Type":"ContainerStarted","Data":"f47f23ab56fb40b496566e8c6286d6ed82f572309928e01110047b513fc3eb73"}
Nov 03 16:12:31.489172 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:31.489136    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b" event={"ID":"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd","Type":"ContainerStarted","Data":"6564b46fcc0696de96ded77857d4a39d1979792d4cd52c77341f09ebf601c541"}
Nov 03 16:12:32.498676 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:32.498634    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b" event={"ID":"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd","Type":"ContainerStarted","Data":"76e57f0f5733aaf6445548378a0fbfcf1d57301f32a694acedebd3f06c537f36"}
Nov 03 16:12:32.499090 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:32.498752    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b" event={"ID":"659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd","Type":"ContainerStarted","Data":"c75c85615e0b6b96dea25c103a81964899cf4a1c85837828579d7e5eebbf1fad"}
Nov 03 16:12:32.709375 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:32.709315    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-84f46ccbf-fkr8b" podStartSLOduration=2.2917577 podStartE2EDuration="5.709299752s" totalImagesPullingTime="3.417542052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:12:32.662336085 +0000 UTC m=+38.486787357" watchObservedRunningTime="2026-11-03 16:12:32.709299752 +0000 UTC m=+38.533751026"
Nov 03 16:12:33.143180 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.143143    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6ffb7c7545-gdbm8"]
Nov 03 16:12:33.145840 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.145815    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.203861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.203826    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ffb7c7545-gdbm8"]
Nov 03 16:12:33.224368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.224324    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-serving-cert\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.224569 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.224387    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8261413-7012-4dae-a220-bb27545ff4c6-tmp\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.224569 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.224524    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-oauth-config\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.224569 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.224554    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6kxz\" (UniqueName: \"kubernetes.io/projected/f8261413-7012-4dae-a220-bb27545ff4c6-kube-api-access-w6kxz\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.224707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.224588    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-service-ca\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.224707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.224617    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-console-config\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.224707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.224654    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-oauth-serving-cert\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.224878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.224779    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-trusted-ca-bundle\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.325682 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.325638    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-trusted-ca-bundle\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.325917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.325753    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-serving-cert\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.325917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.325800    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8261413-7012-4dae-a220-bb27545ff4c6-tmp\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.325917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.325834    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-oauth-config\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.325917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.325862    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6kxz\" (UniqueName: \"kubernetes.io/projected/f8261413-7012-4dae-a220-bb27545ff4c6-kube-api-access-w6kxz\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.325917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.325898    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-service-ca\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.326182 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.325934    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-console-config\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.326182 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.325958    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-oauth-serving-cert\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.326679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.326621    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-service-ca\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.326924 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.326896    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-trusted-ca-bundle\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.327051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.326925    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-oauth-serving-cert\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.327051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.326935    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-console-config\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.327051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.327044    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8261413-7012-4dae-a220-bb27545ff4c6-tmp\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.328710 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.328685    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-serving-cert\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.329124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.329100    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-oauth-config\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.386662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.386625    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6kxz\" (UniqueName: \"kubernetes.io/projected/f8261413-7012-4dae-a220-bb27545ff4c6-kube-api-access-w6kxz\") pod \"console-6ffb7c7545-gdbm8\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") " pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.395780 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:12:33.395688    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:12:33.455852 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.455814    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:33.825298 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:33.825219    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ffb7c7545-gdbm8"]
Nov 03 16:12:33.828652 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:12:33.828623    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8261413_7012_4dae_a220_bb27545ff4c6.slice/crio-1561287fb2d5aeb1329062ae07d133641ebef62e5799fffab61cd7fde6934c90 WatchSource:0}: Error finding container 1561287fb2d5aeb1329062ae07d133641ebef62e5799fffab61cd7fde6934c90: Status 404 returned error can't find the container with id 1561287fb2d5aeb1329062ae07d133641ebef62e5799fffab61cd7fde6934c90
Nov 03 16:12:34.509798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:34.509763    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ffb7c7545-gdbm8" event={"ID":"f8261413-7012-4dae-a220-bb27545ff4c6","Type":"ContainerStarted","Data":"1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb"}
Nov 03 16:12:34.509970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:34.509826    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ffb7c7545-gdbm8" event={"ID":"f8261413-7012-4dae-a220-bb27545ff4c6","Type":"ContainerStarted","Data":"1561287fb2d5aeb1329062ae07d133641ebef62e5799fffab61cd7fde6934c90"}
Nov 03 16:12:34.623533 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:34.623481    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6ffb7c7545-gdbm8" podStartSLOduration=1.623465884 podStartE2EDuration="1.623465884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:12:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:12:34.622972283 +0000 UTC m=+40.447423555" watchObservedRunningTime="2026-11-03 16:12:34.623465884 +0000 UTC m=+40.447917154"
Nov 03 16:12:41.639443 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.639411    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ffb7c7545-gdbm8"]
Nov 03 16:12:41.685540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.685515    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-77845b6847-bjmtx"]
Nov 03 16:12:41.687873 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.687856    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.735918 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.735871    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77845b6847-bjmtx"]
Nov 03 16:12:41.794185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.794132    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae8634a7-c432-4ad3-a7a9-f457e936fda2-console-serving-cert\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.794185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.794187    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-console-config\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.794436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.794262    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae8634a7-c432-4ad3-a7a9-f457e936fda2-console-oauth-config\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.794436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.794307    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x74kc\" (UniqueName: \"kubernetes.io/projected/ae8634a7-c432-4ad3-a7a9-f457e936fda2-kube-api-access-x74kc\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.794436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.794411    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae8634a7-c432-4ad3-a7a9-f457e936fda2-tmp\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.794532 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.794444    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-service-ca\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.794532 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.794475    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-trusted-ca-bundle\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.794532 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.794500    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-oauth-serving-cert\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.895181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.895095    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae8634a7-c432-4ad3-a7a9-f457e936fda2-console-serving-cert\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.895181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.895147    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-console-config\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.895181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.895167    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae8634a7-c432-4ad3-a7a9-f457e936fda2-console-oauth-config\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.895454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.895189    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x74kc\" (UniqueName: \"kubernetes.io/projected/ae8634a7-c432-4ad3-a7a9-f457e936fda2-kube-api-access-x74kc\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.895454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.895229    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae8634a7-c432-4ad3-a7a9-f457e936fda2-tmp\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.895454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.895248    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-service-ca\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.895454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.895272    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-trusted-ca-bundle\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.895454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.895294    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-oauth-serving-cert\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.896186 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.896158    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae8634a7-c432-4ad3-a7a9-f457e936fda2-tmp\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.896299 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.896224    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-console-config\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.896346 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.896323    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-trusted-ca-bundle\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.896536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.896518    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-oauth-serving-cert\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.896658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.896636    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8634a7-c432-4ad3-a7a9-f457e936fda2-service-ca\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.897837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.897816    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae8634a7-c432-4ad3-a7a9-f457e936fda2-console-oauth-config\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.897956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.897939    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae8634a7-c432-4ad3-a7a9-f457e936fda2-console-serving-cert\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.950409 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.950388    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x74kc\" (UniqueName: \"kubernetes.io/projected/ae8634a7-c432-4ad3-a7a9-f457e936fda2-kube-api-access-x74kc\") pod \"console-77845b6847-bjmtx\" (UID: \"ae8634a7-c432-4ad3-a7a9-f457e936fda2\") " pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:41.998393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:41.998355    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:42.368446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:42.368410    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77845b6847-bjmtx"]
Nov 03 16:12:42.372187 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:12:42.372159    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8634a7_c432_4ad3_a7a9_f457e936fda2.slice/crio-ad7cba4c71e7f2d4fc79a885b22218e278c44224a118bb861f3a6511b6ca94bb WatchSource:0}: Error finding container ad7cba4c71e7f2d4fc79a885b22218e278c44224a118bb861f3a6511b6ca94bb: Status 404 returned error can't find the container with id ad7cba4c71e7f2d4fc79a885b22218e278c44224a118bb861f3a6511b6ca94bb
Nov 03 16:12:42.538946 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:42.538910    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77845b6847-bjmtx" event={"ID":"ae8634a7-c432-4ad3-a7a9-f457e936fda2","Type":"ContainerStarted","Data":"7d1cad126c5db3cb8bd5587bdcd2fd5dd33c25f90b8680dceb09010e6ba4b575"}
Nov 03 16:12:42.539123 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:42.538969    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77845b6847-bjmtx" event={"ID":"ae8634a7-c432-4ad3-a7a9-f457e936fda2","Type":"ContainerStarted","Data":"ad7cba4c71e7f2d4fc79a885b22218e278c44224a118bb861f3a6511b6ca94bb"}
Nov 03 16:12:42.651126 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:42.651021    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-77845b6847-bjmtx" podStartSLOduration=1.651007659 podStartE2EDuration="1.651007659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:12:42.650403627 +0000 UTC m=+48.474854898" watchObservedRunningTime="2026-11-03 16:12:42.651007659 +0000 UTC m=+48.475458927"
Nov 03 16:12:43.456850 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:43.456810    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:12:51.998940 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:51.998890    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:51.999366 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:51.999004    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:52.003627 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:52.003604    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:52.573991 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:52.573962    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77845b6847-bjmtx"
Nov 03 16:12:52.733926 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:12:52.733889    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8b6c46946-f7klz"]
Nov 03 16:13:03.390682 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:13:03.390639    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:13:06.658637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.658602    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6ffb7c7545-gdbm8" podUID="f8261413-7012-4dae-a220-bb27545ff4c6" containerName="console" containerID="cri-o://1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb" gracePeriod=15
Nov 03 16:13:06.918507 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.918450    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:13:06.997607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.997580    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-service-ca\") pod \"f8261413-7012-4dae-a220-bb27545ff4c6\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") "
Nov 03 16:13:06.997716 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.997630    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-trusted-ca-bundle\") pod \"f8261413-7012-4dae-a220-bb27545ff4c6\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") "
Nov 03 16:13:06.997716 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.997668    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-serving-cert\") pod \"f8261413-7012-4dae-a220-bb27545ff4c6\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") "
Nov 03 16:13:06.997716 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.997688    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8261413-7012-4dae-a220-bb27545ff4c6-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8261413-7012-4dae-a220-bb27545ff4c6-tmp\") pod \"f8261413-7012-4dae-a220-bb27545ff4c6\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") "
Nov 03 16:13:06.997896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.997720    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-console-config\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-console-config\") pod \"f8261413-7012-4dae-a220-bb27545ff4c6\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") "
Nov 03 16:13:06.997896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.997762    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8261413-7012-4dae-a220-bb27545ff4c6-kube-api-access-w6kxz\" (UniqueName: \"kubernetes.io/projected/f8261413-7012-4dae-a220-bb27545ff4c6-kube-api-access-w6kxz\") pod \"f8261413-7012-4dae-a220-bb27545ff4c6\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") "
Nov 03 16:13:06.997896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.997780    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-oauth-serving-cert\") pod \"f8261413-7012-4dae-a220-bb27545ff4c6\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") "
Nov 03 16:13:06.997896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.997812    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-oauth-config\") pod \"f8261413-7012-4dae-a220-bb27545ff4c6\" (UID: \"f8261413-7012-4dae-a220-bb27545ff4c6\") "
Nov 03 16:13:06.998079 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.997990    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-service-ca" pod "f8261413-7012-4dae-a220-bb27545ff4c6" (UID: "f8261413-7012-4dae-a220-bb27545ff4c6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:13:06.998145 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.998118    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8261413-7012-4dae-a220-bb27545ff4c6-tmp" pod "f8261413-7012-4dae-a220-bb27545ff4c6" (UID: "f8261413-7012-4dae-a220-bb27545ff4c6"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:13:06.998263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.998153    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-trusted-ca-bundle" pod "f8261413-7012-4dae-a220-bb27545ff4c6" (UID: "f8261413-7012-4dae-a220-bb27545ff4c6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:13:06.998449 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.998421    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-console-config" pod "f8261413-7012-4dae-a220-bb27545ff4c6" (UID: "f8261413-7012-4dae-a220-bb27545ff4c6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:13:06.998449 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.998417    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-oauth-serving-cert" pod "f8261413-7012-4dae-a220-bb27545ff4c6" (UID: "f8261413-7012-4dae-a220-bb27545ff4c6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:13:07.000024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:06.999999    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-serving-cert" pod "f8261413-7012-4dae-a220-bb27545ff4c6" (UID: "f8261413-7012-4dae-a220-bb27545ff4c6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:13:07.000160 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.000144    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-oauth-config" pod "f8261413-7012-4dae-a220-bb27545ff4c6" (UID: "f8261413-7012-4dae-a220-bb27545ff4c6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:13:07.000247 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.000226    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8261413-7012-4dae-a220-bb27545ff4c6-kube-api-access-w6kxz" pod "f8261413-7012-4dae-a220-bb27545ff4c6" (UID: "f8261413-7012-4dae-a220-bb27545ff4c6"). InnerVolumeSpecName "kube-api-access-w6kxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:13:07.098334 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.098302    5605 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:07.098334 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.098331    5605 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-trusted-ca-bundle\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:07.098334 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.098342    5605 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-serving-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:07.098556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.098351    5605 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8261413-7012-4dae-a220-bb27545ff4c6-tmp\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:07.098556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.098360    5605 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-console-config\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:07.098556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.098369    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6kxz\" (UniqueName: \"kubernetes.io/projected/f8261413-7012-4dae-a220-bb27545ff4c6-kube-api-access-w6kxz\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:07.098556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.098379    5605 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8261413-7012-4dae-a220-bb27545ff4c6-oauth-serving-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:07.098556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.098387    5605 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8261413-7012-4dae-a220-bb27545ff4c6-console-oauth-config\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:07.624500 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.624460    5605 generic.go:356] "Generic (PLEG): container finished" podID="f8261413-7012-4dae-a220-bb27545ff4c6" containerID="1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb" exitCode=0
Nov 03 16:13:07.624662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.624507    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ffb7c7545-gdbm8"
Nov 03 16:13:07.624662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.624524    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ffb7c7545-gdbm8" event={"ID":"f8261413-7012-4dae-a220-bb27545ff4c6","Type":"ContainerDied","Data":"1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb"}
Nov 03 16:13:07.624662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.624564    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ffb7c7545-gdbm8" event={"ID":"f8261413-7012-4dae-a220-bb27545ff4c6","Type":"ContainerDied","Data":"1561287fb2d5aeb1329062ae07d133641ebef62e5799fffab61cd7fde6934c90"}
Nov 03 16:13:07.624662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.624582    5605 scope.go:122] "RemoveContainer" containerID="1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb"
Nov 03 16:13:07.632607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.632583    5605 scope.go:122] "RemoveContainer" containerID="1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb"
Nov 03 16:13:07.632894 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:13:07.632874    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb\": container with ID starting with 1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb not found: ID does not exist" containerID="1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb"
Nov 03 16:13:07.632948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.632904    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb"} err="failed to get container status \"1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb\": rpc error: code = NotFound desc = could not find container \"1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb\": container with ID starting with 1e100c868c5b0f2ad49fbac30bbece2910e66caa51d94e5d6a12c7eea292f8bb not found: ID does not exist"
Nov 03 16:13:07.783528 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.783498    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ffb7c7545-gdbm8"]
Nov 03 16:13:07.829848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:07.829778    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6ffb7c7545-gdbm8"]
Nov 03 16:13:08.306074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:08.306044    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8261413-7012-4dae-a220-bb27545ff4c6" path="/var/lib/kubelet/pods/f8261413-7012-4dae-a220-bb27545ff4c6/volumes"
Nov 03 16:13:17.753906 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:17.753848    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-8b6c46946-f7klz" podUID="76785a65-390d-4367-ac2b-6bf096c6b18d" containerName="console" containerID="cri-o://a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f" gracePeriod=15
Nov 03 16:13:18.007655 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.007600    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:13:18.078302 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078268    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76785a65-390d-4367-ac2b-6bf096c6b18d-tmp\" (UniqueName: \"kubernetes.io/empty-dir/76785a65-390d-4367-ac2b-6bf096c6b18d-tmp\") pod \"76785a65-390d-4367-ac2b-6bf096c6b18d\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") "
Nov 03 16:13:18.078483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078314    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-console-config\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-console-config\") pod \"76785a65-390d-4367-ac2b-6bf096c6b18d\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") "
Nov 03 16:13:18.078483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078344    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-serving-cert\") pod \"76785a65-390d-4367-ac2b-6bf096c6b18d\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") "
Nov 03 16:13:18.078483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078387    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-oauth-config\") pod \"76785a65-390d-4367-ac2b-6bf096c6b18d\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") "
Nov 03 16:13:18.078483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078421    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-oauth-serving-cert\") pod \"76785a65-390d-4367-ac2b-6bf096c6b18d\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") "
Nov 03 16:13:18.078483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078446    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-trusted-ca-bundle\") pod \"76785a65-390d-4367-ac2b-6bf096c6b18d\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") "
Nov 03 16:13:18.078483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078480    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76785a65-390d-4367-ac2b-6bf096c6b18d-kube-api-access-4djl6\" (UniqueName: \"kubernetes.io/projected/76785a65-390d-4367-ac2b-6bf096c6b18d-kube-api-access-4djl6\") pod \"76785a65-390d-4367-ac2b-6bf096c6b18d\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") "
Nov 03 16:13:18.078820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078518    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-service-ca\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-service-ca\") pod \"76785a65-390d-4367-ac2b-6bf096c6b18d\" (UID: \"76785a65-390d-4367-ac2b-6bf096c6b18d\") "
Nov 03 16:13:18.078820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078548    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76785a65-390d-4367-ac2b-6bf096c6b18d-tmp" pod "76785a65-390d-4367-ac2b-6bf096c6b18d" (UID: "76785a65-390d-4367-ac2b-6bf096c6b18d"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:13:18.078820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078773    5605 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76785a65-390d-4367-ac2b-6bf096c6b18d-tmp\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:18.078977 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078907    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-trusted-ca-bundle" pod "76785a65-390d-4367-ac2b-6bf096c6b18d" (UID: "76785a65-390d-4367-ac2b-6bf096c6b18d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:13:18.078977 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078914    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-oauth-serving-cert" pod "76785a65-390d-4367-ac2b-6bf096c6b18d" (UID: "76785a65-390d-4367-ac2b-6bf096c6b18d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:13:18.078977 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.078952    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-service-ca" pod "76785a65-390d-4367-ac2b-6bf096c6b18d" (UID: "76785a65-390d-4367-ac2b-6bf096c6b18d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:13:18.079088 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.079068    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-console-config" pod "76785a65-390d-4367-ac2b-6bf096c6b18d" (UID: "76785a65-390d-4367-ac2b-6bf096c6b18d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:13:18.080853 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.080831    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-serving-cert" pod "76785a65-390d-4367-ac2b-6bf096c6b18d" (UID: "76785a65-390d-4367-ac2b-6bf096c6b18d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:13:18.081297 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.081276    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-oauth-config" pod "76785a65-390d-4367-ac2b-6bf096c6b18d" (UID: "76785a65-390d-4367-ac2b-6bf096c6b18d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:13:18.081367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.081308    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76785a65-390d-4367-ac2b-6bf096c6b18d-kube-api-access-4djl6" pod "76785a65-390d-4367-ac2b-6bf096c6b18d" (UID: "76785a65-390d-4367-ac2b-6bf096c6b18d"). InnerVolumeSpecName "kube-api-access-4djl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:13:18.179536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.179501    5605 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:18.179536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.179531    5605 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-console-config\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:18.179536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.179543    5605 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-serving-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:18.179784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.179552    5605 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76785a65-390d-4367-ac2b-6bf096c6b18d-console-oauth-config\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:18.179784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.179562    5605 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-oauth-serving-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:18.179784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.179571    5605 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76785a65-390d-4367-ac2b-6bf096c6b18d-trusted-ca-bundle\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:18.179784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.179579    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4djl6\" (UniqueName: \"kubernetes.io/projected/76785a65-390d-4367-ac2b-6bf096c6b18d-kube-api-access-4djl6\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:13:18.666751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.666692    5605 generic.go:356] "Generic (PLEG): container finished" podID="76785a65-390d-4367-ac2b-6bf096c6b18d" containerID="a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f" exitCode=0
Nov 03 16:13:18.666961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.666763    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b6c46946-f7klz"
Nov 03 16:13:18.666961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.666772    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b6c46946-f7klz" event={"ID":"76785a65-390d-4367-ac2b-6bf096c6b18d","Type":"ContainerDied","Data":"a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f"}
Nov 03 16:13:18.666961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.666827    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b6c46946-f7klz" event={"ID":"76785a65-390d-4367-ac2b-6bf096c6b18d","Type":"ContainerDied","Data":"99f565798e7ad8e99005e421011b93c887c108cb824c7635b668e019284a8056"}
Nov 03 16:13:18.666961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.666854    5605 scope.go:122] "RemoveContainer" containerID="a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f"
Nov 03 16:13:18.674465 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.674446    5605 scope.go:122] "RemoveContainer" containerID="a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f"
Nov 03 16:13:18.674764 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:13:18.674745    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f\": container with ID starting with a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f not found: ID does not exist" containerID="a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f"
Nov 03 16:13:18.674833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.674778    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f"} err="failed to get container status \"a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f\": rpc error: code = NotFound desc = could not find container \"a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f\": container with ID starting with a4a097ba423b7a6d8890d826cfc6736b3ec8957e51a4aeba0e9c3057307eb86f not found: ID does not exist"
Nov 03 16:13:18.824544 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.824514    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8b6c46946-f7klz"]
Nov 03 16:13:18.871155 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:18.871122    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8b6c46946-f7klz"]
Nov 03 16:13:20.306085 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:13:20.306045    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76785a65-390d-4367-ac2b-6bf096c6b18d" path="/var/lib/kubelet/pods/76785a65-390d-4367-ac2b-6bf096c6b18d/volumes"
Nov 03 16:13:33.390747 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:13:33.390700    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:14:03.390334 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:14:03.390254    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:14:33.391143 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:14:33.391106    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:14:33.712432 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.712343    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"]
Nov 03 16:14:33.712702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.712684    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f8261413-7012-4dae-a220-bb27545ff4c6" containerName="console"
Nov 03 16:14:33.712782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.712707    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8261413-7012-4dae-a220-bb27545ff4c6" containerName="console"
Nov 03 16:14:33.712782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.712740    5605 cpu_manager.go:414] "containerMap: removing container" podUID="76785a65-390d-4367-ac2b-6bf096c6b18d" containerName="console"
Nov 03 16:14:33.712782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.712746    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76785a65-390d-4367-ac2b-6bf096c6b18d" containerName="console"
Nov 03 16:14:33.712886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.712798    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="76785a65-390d-4367-ac2b-6bf096c6b18d" containerName="console"
Nov 03 16:14:33.712886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.712810    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8261413-7012-4dae-a220-bb27545ff4c6" containerName="console"
Nov 03 16:14:33.715793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.715776    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:33.766707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.766674    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-h5mw9\""
Nov 03 16:14:33.794062 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.794031    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 03 16:14:33.794219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.794078    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 03 16:14:33.812622 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.812586    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"]
Nov 03 16:14:33.814589 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.814560    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:33.814589 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.814594    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:33.814814 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.814681    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsp88\" (UniqueName: \"kubernetes.io/projected/cde29651-78b0-46da-b49b-c1dd085c660d-kube-api-access-rsp88\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:33.821008 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.820983    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"]
Nov 03 16:14:33.824082 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.824067    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:33.875214 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.875184    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 03 16:14:33.875449 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.875433    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 03 16:14:33.877791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.877767    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-57mjv\""
Nov 03 16:14:33.889091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.889068    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 03 16:14:33.889181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.889170    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 03 16:14:33.899656 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.899632    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"]
Nov 03 16:14:33.915328 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915300    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/070e107d-c646-4b95-a962-1ad292bd3b17-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:33.915448 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915338    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsp88\" (UniqueName: \"kubernetes.io/projected/cde29651-78b0-46da-b49b-c1dd085c660d-kube-api-access-rsp88\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:33.915448 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915362    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:33.915555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915479    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:33.915555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915529    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:33.915655 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915589    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:33.915655 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915620    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:33.915655 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915642    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:33.915820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915666    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:33.915820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915684    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggp89\" (UniqueName: \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-kube-api-access-ggp89\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:33.915987 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915968    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:33.916044 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.915985    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:33.971076 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.971009    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsp88\" (UniqueName: \"kubernetes.io/projected/cde29651-78b0-46da-b49b-c1dd085c660d-kube-api-access-rsp88\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:33.976060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.976037    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 03 16:14:33.976060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:33.976049    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 03 16:14:34.016418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.016381    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.016564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.016422    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.016564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.016455    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.016564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.016479    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggp89\" (UniqueName: \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-kube-api-access-ggp89\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.016564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.016513    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/070e107d-c646-4b95-a962-1ad292bd3b17-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.016564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.016549    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.016848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.016598    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.017141 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.017117    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.019119 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.019094    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.019223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.019157    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.019223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.019197    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/070e107d-c646-4b95-a962-1ad292bd3b17-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.019326 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.019256    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.031129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.031109    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:14:34.072539 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.072514    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.072743 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.072708    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggp89\" (UniqueName: \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-kube-api-access-ggp89\") pod \"istiod-openshift-gateway-7876bfcb74-jznfl\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.135249 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.135211    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:34.397913 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.397880    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"]
Nov 03 16:14:34.402344 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:14:34.402311    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde29651_78b0_46da_b49b_c1dd085c660d.slice/crio-25a1a1272cd59535acb66aed7ccb1aa80e5cc111febb0e4c09e0fd15ea9fdcd2 WatchSource:0}: Error finding container 25a1a1272cd59535acb66aed7ccb1aa80e5cc111febb0e4c09e0fd15ea9fdcd2: Status 404 returned error can't find the container with id 25a1a1272cd59535acb66aed7ccb1aa80e5cc111febb0e4c09e0fd15ea9fdcd2
Nov 03 16:14:34.509235 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.509197    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"]
Nov 03 16:14:34.512702 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:14:34.512674    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod070e107d_c646_4b95_a962_1ad292bd3b17.slice/crio-710780ada957b94055ebc2b3f551ea6669ecc9075a31e2ea633a3322a5757a53 WatchSource:0}: Error finding container 710780ada957b94055ebc2b3f551ea6669ecc9075a31e2ea633a3322a5757a53: Status 404 returned error can't find the container with id 710780ada957b94055ebc2b3f551ea6669ecc9075a31e2ea633a3322a5757a53
Nov 03 16:14:34.918432 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.918387    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl" event={"ID":"070e107d-c646-4b95-a962-1ad292bd3b17","Type":"ContainerStarted","Data":"710780ada957b94055ebc2b3f551ea6669ecc9075a31e2ea633a3322a5757a53"}
Nov 03 16:14:34.919617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:34.919585    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286" event={"ID":"cde29651-78b0-46da-b49b-c1dd085c660d","Type":"ContainerStarted","Data":"25a1a1272cd59535acb66aed7ccb1aa80e5cc111febb0e4c09e0fd15ea9fdcd2"}
Nov 03 16:14:40.398535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:40.398099    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:14:40.398535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:40.398159    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:14:40.944329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:40.944290    5605 generic.go:356] "Generic (PLEG): container finished" podID="cde29651-78b0-46da-b49b-c1dd085c660d" containerID="11cf25c0c65545d9f72287f7c24b604fca94f10be9d45a7b7489833fd10830c6" exitCode=0
Nov 03 16:14:40.944528 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:40.944377    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286" event={"ID":"cde29651-78b0-46da-b49b-c1dd085c660d","Type":"ContainerDied","Data":"11cf25c0c65545d9f72287f7c24b604fca94f10be9d45a7b7489833fd10830c6"}
Nov 03 16:14:40.946037 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:40.946008    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl" event={"ID":"070e107d-c646-4b95-a962-1ad292bd3b17","Type":"ContainerStarted","Data":"27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93"}
Nov 03 16:14:40.946225 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:40.946177    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:41.170416 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:41.170360    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl" podStartSLOduration=2.287492911 podStartE2EDuration="8.170343597s" totalImagesPullingTime="5.882850686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:14:41.16915439 +0000 UTC m=+166.993605662" watchObservedRunningTime="2026-11-03 16:14:41.170343597 +0000 UTC m=+166.994794862"
Nov 03 16:14:41.951451 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:41.951419    5605 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-jznfl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:14:41.951927 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:41.951475    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl" podUID="070e107d-c646-4b95-a962-1ad292bd3b17" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:14:44.950860 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:44.950828    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:14:45.966632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:45.966594    5605 generic.go:356] "Generic (PLEG): container finished" podID="cde29651-78b0-46da-b49b-c1dd085c660d" containerID="e62fb3debdf21bb48cc31a1a4d8c863f224b980de23713dd34b6ac82550317a4" exitCode=0
Nov 03 16:14:45.967109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:14:45.966690    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286" event={"ID":"cde29651-78b0-46da-b49b-c1dd085c660d","Type":"ContainerDied","Data":"e62fb3debdf21bb48cc31a1a4d8c863f224b980de23713dd34b6ac82550317a4"}
Nov 03 16:15:02.028568 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:02.028473    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286" event={"ID":"cde29651-78b0-46da-b49b-c1dd085c660d","Type":"ContainerStarted","Data":"39488546169a95126ff9b16d9f4dc79773f65b2f12104f824b05d1b34b8c7e04"}
Nov 03 16:15:03.034392 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:03.034362    5605 generic.go:356] "Generic (PLEG): container finished" podID="cde29651-78b0-46da-b49b-c1dd085c660d" containerID="39488546169a95126ff9b16d9f4dc79773f65b2f12104f824b05d1b34b8c7e04" exitCode=0
Nov 03 16:15:03.034800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:03.034413    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286" event={"ID":"cde29651-78b0-46da-b49b-c1dd085c660d","Type":"ContainerDied","Data":"39488546169a95126ff9b16d9f4dc79773f65b2f12104f824b05d1b34b8c7e04"}
Nov 03 16:15:03.391117 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:15:03.391077    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:15:04.163621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:04.163597    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:15:04.268406 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:04.268372    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-bundle\" (UniqueName: \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-bundle\") pod \"cde29651-78b0-46da-b49b-c1dd085c660d\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") "
Nov 03 16:15:04.268567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:04.268445    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-util\" (UniqueName: \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-util\") pod \"cde29651-78b0-46da-b49b-c1dd085c660d\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") "
Nov 03 16:15:04.268567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:04.268500    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cde29651-78b0-46da-b49b-c1dd085c660d-kube-api-access-rsp88\" (UniqueName: \"kubernetes.io/projected/cde29651-78b0-46da-b49b-c1dd085c660d-kube-api-access-rsp88\") pod \"cde29651-78b0-46da-b49b-c1dd085c660d\" (UID: \"cde29651-78b0-46da-b49b-c1dd085c660d\") "
Nov 03 16:15:04.268784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:04.268762    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-bundle" pod "cde29651-78b0-46da-b49b-c1dd085c660d" (UID: "cde29651-78b0-46da-b49b-c1dd085c660d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:15:04.270824 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:04.270798    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde29651-78b0-46da-b49b-c1dd085c660d-kube-api-access-rsp88" pod "cde29651-78b0-46da-b49b-c1dd085c660d" (UID: "cde29651-78b0-46da-b49b-c1dd085c660d"). InnerVolumeSpecName "kube-api-access-rsp88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:15:04.272918 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:04.272897    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-util" pod "cde29651-78b0-46da-b49b-c1dd085c660d" (UID: "cde29651-78b0-46da-b49b-c1dd085c660d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:15:04.370083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:04.370041    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsp88\" (UniqueName: \"kubernetes.io/projected/cde29651-78b0-46da-b49b-c1dd085c660d-kube-api-access-rsp88\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:15:04.370083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:04.370078    5605 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-bundle\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:15:04.370083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:04.370092    5605 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cde29651-78b0-46da-b49b-c1dd085c660d-util\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:15:05.042454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:05.042428    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286"
Nov 03 16:15:05.042617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:05.042422    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e6448b286" event={"ID":"cde29651-78b0-46da-b49b-c1dd085c660d","Type":"ContainerDied","Data":"25a1a1272cd59535acb66aed7ccb1aa80e5cc111febb0e4c09e0fd15ea9fdcd2"}
Nov 03 16:15:05.042617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:05.042544    5605 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25a1a1272cd59535acb66aed7ccb1aa80e5cc111febb0e4c09e0fd15ea9fdcd2"
Nov 03 16:15:08.993685 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:08.993650    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"]
Nov 03 16:15:08.994078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:08.993948    5605 cpu_manager.go:414] "containerMap: removing container" podUID="cde29651-78b0-46da-b49b-c1dd085c660d" containerName="pull"
Nov 03 16:15:08.994078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:08.993959    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cde29651-78b0-46da-b49b-c1dd085c660d" containerName="pull"
Nov 03 16:15:08.994078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:08.993979    5605 cpu_manager.go:414] "containerMap: removing container" podUID="cde29651-78b0-46da-b49b-c1dd085c660d" containerName="util"
Nov 03 16:15:08.994078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:08.993988    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cde29651-78b0-46da-b49b-c1dd085c660d" containerName="util"
Nov 03 16:15:08.994078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:08.994004    5605 cpu_manager.go:414] "containerMap: removing container" podUID="cde29651-78b0-46da-b49b-c1dd085c660d" containerName="extract"
Nov 03 16:15:08.994078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:08.994009    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cde29651-78b0-46da-b49b-c1dd085c660d" containerName="extract"
Nov 03 16:15:08.994078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:08.994058    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="cde29651-78b0-46da-b49b-c1dd085c660d" containerName="extract"
Nov 03 16:15:08.996135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:08.996111    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.096535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.096493    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 03 16:15:09.096861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.096839    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 03 16:15:09.096970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.096916    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 03 16:15:09.096970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.096918    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-rbhfv\""
Nov 03 16:15:09.097818 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.097804    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 03 16:15:09.097959 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.097947    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 03 16:15:09.106030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.106000    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"]
Nov 03 16:15:09.106291 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.106265    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9jxh\" (UniqueName: \"kubernetes.io/projected/fed21f2b-6944-4fb7-8e37-87a916d2b11c-kube-api-access-c9jxh\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.106383 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.106305    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fed21f2b-6944-4fb7-8e37-87a916d2b11c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.106383 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.106338    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fed21f2b-6944-4fb7-8e37-87a916d2b11c-tmp\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.106486 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.106387    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fed21f2b-6944-4fb7-8e37-87a916d2b11c-metrics-cert\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.106486 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.106466    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fed21f2b-6944-4fb7-8e37-87a916d2b11c-manifests\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.106576 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.106528    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fed21f2b-6944-4fb7-8e37-87a916d2b11c-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.206957 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.206929    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fed21f2b-6944-4fb7-8e37-87a916d2b11c-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.207164 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.206973    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jxh\" (UniqueName: \"kubernetes.io/projected/fed21f2b-6944-4fb7-8e37-87a916d2b11c-kube-api-access-c9jxh\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.207164 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.207004    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fed21f2b-6944-4fb7-8e37-87a916d2b11c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.207164 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.207037    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fed21f2b-6944-4fb7-8e37-87a916d2b11c-tmp\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.207164 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.207056    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fed21f2b-6944-4fb7-8e37-87a916d2b11c-metrics-cert\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.207164 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.207077    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fed21f2b-6944-4fb7-8e37-87a916d2b11c-manifests\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.207567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.207539    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fed21f2b-6944-4fb7-8e37-87a916d2b11c-manifests\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.209634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.209602    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fed21f2b-6944-4fb7-8e37-87a916d2b11c-tmp\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.209780 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.209675    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fed21f2b-6944-4fb7-8e37-87a916d2b11c-metrics-cert\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.209780 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.209762    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fed21f2b-6944-4fb7-8e37-87a916d2b11c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.209941 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.209835    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fed21f2b-6944-4fb7-8e37-87a916d2b11c-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.263265 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.263195    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9jxh\" (UniqueName: \"kubernetes.io/projected/fed21f2b-6944-4fb7-8e37-87a916d2b11c-kube-api-access-c9jxh\") pod \"opendatahub-operator-controller-manager-7d94989496-k75w7\" (UID: \"fed21f2b-6944-4fb7-8e37-87a916d2b11c\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.309310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.309283    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:09.798302 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:09.798117    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"]
Nov 03 16:15:09.801345 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:15:09.801319    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed21f2b_6944_4fb7_8e37_87a916d2b11c.slice/crio-e50d4f8b35ee0fc94741d94138cddabb832f5a6448d46125ec47762273046924 WatchSource:0}: Error finding container e50d4f8b35ee0fc94741d94138cddabb832f5a6448d46125ec47762273046924: Status 404 returned error can't find the container with id e50d4f8b35ee0fc94741d94138cddabb832f5a6448d46125ec47762273046924
Nov 03 16:15:10.060976 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:10.060890    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7" event={"ID":"fed21f2b-6944-4fb7-8e37-87a916d2b11c","Type":"ContainerStarted","Data":"e50d4f8b35ee0fc94741d94138cddabb832f5a6448d46125ec47762273046924"}
Nov 03 16:15:12.067564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.066156    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"]
Nov 03 16:15:12.068874 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.068849    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:12.165119 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.165083    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-h5mw9\""
Nov 03 16:15:12.165119 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.165089    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 03 16:15:12.165335 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.165173    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 03 16:15:12.165913 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.165889    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"]
Nov 03 16:15:12.231958 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.231919    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrx8q\" (UniqueName: \"kubernetes.io/projected/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-kube-api-access-mrx8q\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:12.232154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.232009    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:12.232154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.232050    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:12.332614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.332522    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrx8q\" (UniqueName: \"kubernetes.io/projected/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-kube-api-access-mrx8q\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:12.332805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.332616    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:12.332805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.332657    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:12.333023 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.333002    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:12.333092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.333062    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:12.396584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.396535    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrx8q\" (UniqueName: \"kubernetes.io/projected/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-kube-api-access-mrx8q\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:12.681837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:12.681806    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:13.117222 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:13.117148    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"]
Nov 03 16:15:13.121027 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:15:13.121000    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1a3182_c9db_49fb_bc5b_ce2bf0cf04dc.slice/crio-72c32aa714fea7b85a27dd04da740e7a2451f6cdb9eda5ee1d1e170b7e0a2383 WatchSource:0}: Error finding container 72c32aa714fea7b85a27dd04da740e7a2451f6cdb9eda5ee1d1e170b7e0a2383: Status 404 returned error can't find the container with id 72c32aa714fea7b85a27dd04da740e7a2451f6cdb9eda5ee1d1e170b7e0a2383
Nov 03 16:15:14.077443 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:14.077406    5605 generic.go:356] "Generic (PLEG): container finished" podID="ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" containerID="51a608b922198970a4eeda060841932353b24373a58dd2efc3116fb766f0a873" exitCode=0
Nov 03 16:15:14.077621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:14.077458    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm" event={"ID":"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc","Type":"ContainerDied","Data":"51a608b922198970a4eeda060841932353b24373a58dd2efc3116fb766f0a873"}
Nov 03 16:15:14.077621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:14.077529    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm" event={"ID":"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc","Type":"ContainerStarted","Data":"72c32aa714fea7b85a27dd04da740e7a2451f6cdb9eda5ee1d1e170b7e0a2383"}
Nov 03 16:15:15.082687 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:15.082655    5605 generic.go:356] "Generic (PLEG): container finished" podID="ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" containerID="4792366a5ff338ab5fc848c7e32adf6d99e5214d2a2424dea851a5e120721985" exitCode=0
Nov 03 16:15:15.083078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:15.082696    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm" event={"ID":"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc","Type":"ContainerDied","Data":"4792366a5ff338ab5fc848c7e32adf6d99e5214d2a2424dea851a5e120721985"}
Nov 03 16:15:16.089588 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:16.089552    5605 generic.go:356] "Generic (PLEG): container finished" podID="ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" containerID="be54b5e1b604dbbad5d61e8634a31402e01ae87384d0cb0994acd2b67d492203" exitCode=0
Nov 03 16:15:16.090046 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:16.089643    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm" event={"ID":"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc","Type":"ContainerDied","Data":"be54b5e1b604dbbad5d61e8634a31402e01ae87384d0cb0994acd2b67d492203"}
Nov 03 16:15:16.091272 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:16.091250    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7" event={"ID":"fed21f2b-6944-4fb7-8e37-87a916d2b11c","Type":"ContainerStarted","Data":"f7b7eaf391d5102c7a757dc5e96d2911b454fce4e84136851a1ec7136a8aec7c"}
Nov 03 16:15:17.096842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.096792    5605 generic.go:356] "Generic (PLEG): container finished" podID="fed21f2b-6944-4fb7-8e37-87a916d2b11c" containerID="f7b7eaf391d5102c7a757dc5e96d2911b454fce4e84136851a1ec7136a8aec7c" exitCode=0
Nov 03 16:15:17.097321 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.096882    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7" event={"ID":"fed21f2b-6944-4fb7-8e37-87a916d2b11c","Type":"ContainerDied","Data":"f7b7eaf391d5102c7a757dc5e96d2911b454fce4e84136851a1ec7136a8aec7c"}
Nov 03 16:15:17.220259 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.220228    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:17.378663 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.378572    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-kube-api-access-mrx8q\" (UniqueName: \"kubernetes.io/projected/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-kube-api-access-mrx8q\") pod \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") "
Nov 03 16:15:17.378663 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.378626    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-util\" (UniqueName: \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-util\") pod \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") "
Nov 03 16:15:17.378911 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.378685    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-bundle\") pod \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\" (UID: \"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc\") "
Nov 03 16:15:17.379085 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.379060    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-bundle" pod "ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" (UID: "ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:15:17.380981 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.380960    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-kube-api-access-mrx8q" pod "ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" (UID: "ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc"). InnerVolumeSpecName "kube-api-access-mrx8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:15:17.383642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.383621    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-util" pod "ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" (UID: "ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:15:17.480038 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.480009    5605 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-util\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:15:17.480038 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.480035    5605 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-bundle\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:15:17.480038 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:17.480045    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrx8q\" (UniqueName: \"kubernetes.io/projected/ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc-kube-api-access-mrx8q\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:15:18.102378 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:18.102337    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7" event={"ID":"fed21f2b-6944-4fb7-8e37-87a916d2b11c","Type":"ContainerStarted","Data":"b842ac62808b86f09d7990998217980cfdebd31839d3771b4c688f8f3e728390"}
Nov 03 16:15:18.102970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:18.102459    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:18.104307 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:18.104279    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm" event={"ID":"ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc","Type":"ContainerDied","Data":"72c32aa714fea7b85a27dd04da740e7a2451f6cdb9eda5ee1d1e170b7e0a2383"}
Nov 03 16:15:18.104414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:18.104318    5605 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72c32aa714fea7b85a27dd04da740e7a2451f6cdb9eda5ee1d1e170b7e0a2383"
Nov 03 16:15:18.104414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:18.104321    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644r6btm"
Nov 03 16:15:18.287453 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:18.287345    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7" podStartSLOduration=2.802318821 podStartE2EDuration="10.287332033s" totalImagesPullingTime="7.000232028s" totalInitContainerRuntime="484.781184ms" isStatefulPod=false podCreationTimestamp="2026-11-03 16:15:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:15:18.286137213 +0000 UTC m=+204.110588497" watchObservedRunningTime="2026-11-03 16:15:18.287332033 +0000 UTC m=+204.111783297"
Nov 03 16:15:29.110604 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:29.110524    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7d94989496-k75w7"
Nov 03 16:15:33.390334 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:15:33.390299    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:15:47.896875 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.896842    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"]
Nov 03 16:15:47.897320 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.897262    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" containerName="extract"
Nov 03 16:15:47.897320 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.897281    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" containerName="extract"
Nov 03 16:15:47.897320 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.897318    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" containerName="util"
Nov 03 16:15:47.897420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.897325    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" containerName="util"
Nov 03 16:15:47.897420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.897334    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" containerName="pull"
Nov 03 16:15:47.897420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.897343    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" containerName="pull"
Nov 03 16:15:47.897505 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.897456    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed1a3182-c9db-49fb-bc5b-ce2bf0cf04dc" containerName="extract"
Nov 03 16:15:47.903641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.903621    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:47.955338 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.955312    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 03 16:15:47.955509 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:47.955490    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 03 16:15:48.001049 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.001014    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xgr22\""
Nov 03 16:15:48.001817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.001792    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"]
Nov 03 16:15:48.011635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.011600    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b41055d-d1c9-4ba3-b938-42c94b091764-tmp\") pod \"kube-auth-proxy-5fbcdb5d6b-z4mv5\" (UID: \"3b41055d-d1c9-4ba3-b938-42c94b091764\") " pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:48.011796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.011652    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrg9s\" (UniqueName: \"kubernetes.io/projected/3b41055d-d1c9-4ba3-b938-42c94b091764-kube-api-access-nrg9s\") pod \"kube-auth-proxy-5fbcdb5d6b-z4mv5\" (UID: \"3b41055d-d1c9-4ba3-b938-42c94b091764\") " pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:48.011868 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.011838    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b41055d-d1c9-4ba3-b938-42c94b091764-tls-certs\") pod \"kube-auth-proxy-5fbcdb5d6b-z4mv5\" (UID: \"3b41055d-d1c9-4ba3-b938-42c94b091764\") " pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:48.112529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.112487    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b41055d-d1c9-4ba3-b938-42c94b091764-tmp\") pod \"kube-auth-proxy-5fbcdb5d6b-z4mv5\" (UID: \"3b41055d-d1c9-4ba3-b938-42c94b091764\") " pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:48.112529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.112532    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrg9s\" (UniqueName: \"kubernetes.io/projected/3b41055d-d1c9-4ba3-b938-42c94b091764-kube-api-access-nrg9s\") pod \"kube-auth-proxy-5fbcdb5d6b-z4mv5\" (UID: \"3b41055d-d1c9-4ba3-b938-42c94b091764\") " pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:48.112835 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.112581    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b41055d-d1c9-4ba3-b938-42c94b091764-tls-certs\") pod \"kube-auth-proxy-5fbcdb5d6b-z4mv5\" (UID: \"3b41055d-d1c9-4ba3-b938-42c94b091764\") " pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:48.114905 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.114880    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b41055d-d1c9-4ba3-b938-42c94b091764-tmp\") pod \"kube-auth-proxy-5fbcdb5d6b-z4mv5\" (UID: \"3b41055d-d1c9-4ba3-b938-42c94b091764\") " pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:48.115151 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.115131    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b41055d-d1c9-4ba3-b938-42c94b091764-tls-certs\") pod \"kube-auth-proxy-5fbcdb5d6b-z4mv5\" (UID: \"3b41055d-d1c9-4ba3-b938-42c94b091764\") " pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:48.170151 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.170078    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrg9s\" (UniqueName: \"kubernetes.io/projected/3b41055d-d1c9-4ba3-b938-42c94b091764-kube-api-access-nrg9s\") pod \"kube-auth-proxy-5fbcdb5d6b-z4mv5\" (UID: \"3b41055d-d1c9-4ba3-b938-42c94b091764\") " pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:48.214387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.214356    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"
Nov 03 16:15:48.230604 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.230571    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-d85sw"]
Nov 03 16:15:48.232881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.232861    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:15:48.282772 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.282690    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jjqwv\""
Nov 03 16:15:48.283822 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.283801    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 03 16:15:48.283951 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.283882    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 03 16:15:48.283951 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.283921    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 03 16:15:48.289560 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.289536    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-d85sw"]
Nov 03 16:15:48.314554 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.314529    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwqxs\" (UniqueName: \"kubernetes.io/projected/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-kube-api-access-kwqxs\") pod \"limitador-limitador-74dfcfb894-d85sw\" (UID: \"4f4f5a4f-443e-4894-b443-a18ca23eb2b0\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:15:48.314709 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.314659    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-config-file\") pod \"limitador-limitador-74dfcfb894-d85sw\" (UID: \"4f4f5a4f-443e-4894-b443-a18ca23eb2b0\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:15:48.415761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.415709    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwqxs\" (UniqueName: \"kubernetes.io/projected/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-kube-api-access-kwqxs\") pod \"limitador-limitador-74dfcfb894-d85sw\" (UID: \"4f4f5a4f-443e-4894-b443-a18ca23eb2b0\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:15:48.415922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.415905    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-config-file\") pod \"limitador-limitador-74dfcfb894-d85sw\" (UID: \"4f4f5a4f-443e-4894-b443-a18ca23eb2b0\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:15:48.416584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.416557    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-config-file\") pod \"limitador-limitador-74dfcfb894-d85sw\" (UID: \"4f4f5a4f-443e-4894-b443-a18ca23eb2b0\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:15:48.469624 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.469544    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-d85sw"]
Nov 03 16:15:48.473801 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.473767    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwqxs\" (UniqueName: \"kubernetes.io/projected/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-kube-api-access-kwqxs\") pod \"limitador-limitador-74dfcfb894-d85sw\" (UID: \"4f4f5a4f-443e-4894-b443-a18ca23eb2b0\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:15:48.546883 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.546846    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:15:48.584144 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.584065    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5"]
Nov 03 16:15:48.588113 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:15:48.588071    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b41055d_d1c9_4ba3_b938_42c94b091764.slice/crio-908fe01653ce8c7e95d151eafb3b5236d67882ab6c7f7aa929bcde5f0a4e6f7d WatchSource:0}: Error finding container 908fe01653ce8c7e95d151eafb3b5236d67882ab6c7f7aa929bcde5f0a4e6f7d: Status 404 returned error can't find the container with id 908fe01653ce8c7e95d151eafb3b5236d67882ab6c7f7aa929bcde5f0a4e6f7d
Nov 03 16:15:48.914110 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:48.914076    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-d85sw"]
Nov 03 16:15:48.920420 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:15:48.920389    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4f5a4f_443e_4894_b443_a18ca23eb2b0.slice/crio-01aaf72d36c9a1badac090b2fcd40740b35753f6c4befe56ba91de706b68d50f WatchSource:0}: Error finding container 01aaf72d36c9a1badac090b2fcd40740b35753f6c4befe56ba91de706b68d50f: Status 404 returned error can't find the container with id 01aaf72d36c9a1badac090b2fcd40740b35753f6c4befe56ba91de706b68d50f
Nov 03 16:15:49.215874 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:49.215783    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5" event={"ID":"3b41055d-d1c9-4ba3-b938-42c94b091764","Type":"ContainerStarted","Data":"908fe01653ce8c7e95d151eafb3b5236d67882ab6c7f7aa929bcde5f0a4e6f7d"}
Nov 03 16:15:49.216969 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:49.216940    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw" event={"ID":"4f4f5a4f-443e-4894-b443-a18ca23eb2b0","Type":"ContainerStarted","Data":"01aaf72d36c9a1badac090b2fcd40740b35753f6c4befe56ba91de706b68d50f"}
Nov 03 16:15:53.234323 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:53.234288    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw" event={"ID":"4f4f5a4f-443e-4894-b443-a18ca23eb2b0","Type":"ContainerStarted","Data":"a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095"}
Nov 03 16:15:53.234747 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:53.234377    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:15:53.348003 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:53.347783    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw" podStartSLOduration=1.780526125 podStartE2EDuration="5.347765765s" totalImagesPullingTime="3.56723964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:15:53.347224965 +0000 UTC m=+239.171676236" watchObservedRunningTime="2026-11-03 16:15:53.347765765 +0000 UTC m=+239.172217039"
Nov 03 16:15:54.239956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:54.239920    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5" event={"ID":"3b41055d-d1c9-4ba3-b938-42c94b091764","Type":"ContainerStarted","Data":"490d747669fcec4df7f244a4a7382dabd5076e0f34b69a6b64de0a7c00dfcce3"}
Nov 03 16:15:54.351694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:15:54.351634    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5fbcdb5d6b-z4mv5" podStartSLOduration=2.279508057 podStartE2EDuration="7.351618142s" totalImagesPullingTime="5.072110085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:15:54.351464919 +0000 UTC m=+240.175916185" watchObservedRunningTime="2026-11-03 16:15:54.351618142 +0000 UTC m=+240.176069414"
Nov 03 16:16:03.390473 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:16:03.390433    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:16:04.241312 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.241284    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:16:04.884781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.884747    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"]
Nov 03 16:16:04.887691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.887666    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:04.938308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.938268    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 03 16:16:04.955208 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.955177    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7f8x\" (UniqueName: \"kubernetes.io/projected/768d875e-8b77-45a3-9a0d-b26f4c6b0470-kube-api-access-f7f8x\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:04.955368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.955227    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/768d875e-8b77-45a3-9a0d-b26f4c6b0470-manager-config\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:04.955368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.955290    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768d875e-8b77-45a3-9a0d-b26f4c6b0470-metrics-certs\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:04.955368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.955330    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/768d875e-8b77-45a3-9a0d-b26f4c6b0470-cert\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:04.983632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.983602    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 03 16:16:04.983632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.983629    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Nov 03 16:16:04.983950 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.983708    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Nov 03 16:16:04.983950 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.983757    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-pcjpj\""
Nov 03 16:16:04.983950 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.983846    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Nov 03 16:16:04.985440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:04.985423    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"]
Nov 03 16:16:05.056642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:05.056608    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/768d875e-8b77-45a3-9a0d-b26f4c6b0470-cert\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:05.056863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:05.056668    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7f8x\" (UniqueName: \"kubernetes.io/projected/768d875e-8b77-45a3-9a0d-b26f4c6b0470-kube-api-access-f7f8x\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:05.056863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:05.056696    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/768d875e-8b77-45a3-9a0d-b26f4c6b0470-manager-config\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:05.056863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:05.056748    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768d875e-8b77-45a3-9a0d-b26f4c6b0470-metrics-certs\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:05.057469 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:05.057446    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/768d875e-8b77-45a3-9a0d-b26f4c6b0470-manager-config\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:05.059279 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:05.059253    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/768d875e-8b77-45a3-9a0d-b26f4c6b0470-cert\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:05.059377 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:05.059327    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768d875e-8b77-45a3-9a0d-b26f4c6b0470-metrics-certs\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:05.112065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:05.112036    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7f8x\" (UniqueName: \"kubernetes.io/projected/768d875e-8b77-45a3-9a0d-b26f4c6b0470-kube-api-access-f7f8x\") pod \"jobset-controller-manager-69dd86d5f5-7fgq4\" (UID: \"768d875e-8b77-45a3-9a0d-b26f4c6b0470\") " pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:05.199586 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:05.199517    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:05.570069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:05.569992    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"]
Nov 03 16:16:05.573300 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:16:05.573271    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768d875e_8b77_45a3_9a0d_b26f4c6b0470.slice/crio-31db344f202e75860622f48677ff2883d7f34d1ecd464c53545a464b1ef42b45 WatchSource:0}: Error finding container 31db344f202e75860622f48677ff2883d7f34d1ecd464c53545a464b1ef42b45: Status 404 returned error can't find the container with id 31db344f202e75860622f48677ff2883d7f34d1ecd464c53545a464b1ef42b45
Nov 03 16:16:06.285421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:06.285384    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4" event={"ID":"768d875e-8b77-45a3-9a0d-b26f4c6b0470","Type":"ContainerStarted","Data":"31db344f202e75860622f48677ff2883d7f34d1ecd464c53545a464b1ef42b45"}
Nov 03 16:16:07.212431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:07.212393    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-d85sw"]
Nov 03 16:16:07.212672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:07.212640    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw" podUID="4f4f5a4f-443e-4894-b443-a18ca23eb2b0" containerName="limitador" containerID="cri-o://a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095" gracePeriod=30
Nov 03 16:16:07.753639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:07.753617    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:16:07.883034 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:07.883001    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-kube-api-access-kwqxs\" (UniqueName: \"kubernetes.io/projected/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-kube-api-access-kwqxs\") pod \"4f4f5a4f-443e-4894-b443-a18ca23eb2b0\" (UID: \"4f4f5a4f-443e-4894-b443-a18ca23eb2b0\") "
Nov 03 16:16:07.883243 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:07.883101    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-config-file\" (UniqueName: \"kubernetes.io/configmap/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-config-file\") pod \"4f4f5a4f-443e-4894-b443-a18ca23eb2b0\" (UID: \"4f4f5a4f-443e-4894-b443-a18ca23eb2b0\") "
Nov 03 16:16:07.883495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:07.883468    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-config-file" pod "4f4f5a4f-443e-4894-b443-a18ca23eb2b0" (UID: "4f4f5a4f-443e-4894-b443-a18ca23eb2b0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:16:07.885518 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:07.885492    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-kube-api-access-kwqxs" pod "4f4f5a4f-443e-4894-b443-a18ca23eb2b0" (UID: "4f4f5a4f-443e-4894-b443-a18ca23eb2b0"). InnerVolumeSpecName "kube-api-access-kwqxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:16:07.984350 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:07.984322    5605 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-config-file\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:16:07.984350 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:07.984349    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwqxs\" (UniqueName: \"kubernetes.io/projected/4f4f5a4f-443e-4894-b443-a18ca23eb2b0-kube-api-access-kwqxs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:16:08.295896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:08.295814    5605 generic.go:356] "Generic (PLEG): container finished" podID="4f4f5a4f-443e-4894-b443-a18ca23eb2b0" containerID="a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095" exitCode=0
Nov 03 16:16:08.295896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:08.295866    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw"
Nov 03 16:16:08.295896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:08.295877    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw" event={"ID":"4f4f5a4f-443e-4894-b443-a18ca23eb2b0","Type":"ContainerDied","Data":"a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095"}
Nov 03 16:16:08.296112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:08.295920    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-d85sw" event={"ID":"4f4f5a4f-443e-4894-b443-a18ca23eb2b0","Type":"ContainerDied","Data":"01aaf72d36c9a1badac090b2fcd40740b35753f6c4befe56ba91de706b68d50f"}
Nov 03 16:16:08.296112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:08.295941    5605 scope.go:122] "RemoveContainer" containerID="a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095"
Nov 03 16:16:08.305250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:08.305236    5605 scope.go:122] "RemoveContainer" containerID="a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095"
Nov 03 16:16:08.305522 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:16:08.305505    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095\": container with ID starting with a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095 not found: ID does not exist" containerID="a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095"
Nov 03 16:16:08.305572 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:08.305534    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095"} err="failed to get container status \"a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095\": rpc error: code = NotFound desc = could not find container \"a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095\": container with ID starting with a6fe87b684c8d090fe815648957431bdc4552a01d077b7400f98ebc8037de095 not found: ID does not exist"
Nov 03 16:16:08.461075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:08.461042    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-d85sw"]
Nov 03 16:16:08.507511 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:08.507474    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-d85sw"]
Nov 03 16:16:10.306662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:10.306629    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f4f5a4f-443e-4894-b443-a18ca23eb2b0" path="/var/lib/kubelet/pods/4f4f5a4f-443e-4894-b443-a18ca23eb2b0/volumes"
Nov 03 16:16:12.315355 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:12.315320    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4" event={"ID":"768d875e-8b77-45a3-9a0d-b26f4c6b0470","Type":"ContainerStarted","Data":"6c3fece9bf93f8966d18a871b542488f689b740ae6d1dfb781777fbeeb7550bf"}
Nov 03 16:16:12.315805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:12.315438    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:12.426549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:12.426498    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4" podStartSLOduration=2.660665283 podStartE2EDuration="8.426483474s" totalImagesPullingTime="5.765818191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:16:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:16:12.425327459 +0000 UTC m=+258.249778729" watchObservedRunningTime="2026-11-03 16:16:12.426483474 +0000 UTC m=+258.250934722"
Nov 03 16:16:23.324239 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:23.324209    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-69dd86d5f5-7fgq4"
Nov 03 16:16:33.392090 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:16:33.392053    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:16:54.328503 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:54.328477    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:16:54.330378 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:16:54.330358    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:17:03.390627 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:17:03.390544    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:17:17.015708 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.015677    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"]
Nov 03 16:17:17.016216 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.015991    5605 cpu_manager.go:414] "containerMap: removing container" podUID="4f4f5a4f-443e-4894-b443-a18ca23eb2b0" containerName="limitador"
Nov 03 16:17:17.016216 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.016002    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f4f5a4f-443e-4894-b443-a18ca23eb2b0" containerName="limitador"
Nov 03 16:17:17.016216 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.016047    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f4f5a4f-443e-4894-b443-a18ca23eb2b0" containerName="limitador"
Nov 03 16:17:17.018929 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.018910    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.081148 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.081113    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"]
Nov 03 16:17:17.141764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.141711    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b49dafeb-4657-4a77-bcd0-ede25c02a57c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.141933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.141827    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.141933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.141875    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.141933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.141901    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b49dafeb-4657-4a77-bcd0-ede25c02a57c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.142057 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.141997    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cjvv\" (UniqueName: \"kubernetes.io/projected/b49dafeb-4657-4a77-bcd0-ede25c02a57c-kube-api-access-2cjvv\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.142057 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.142029    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.142144 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.142082    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.243194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.243159    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.243359 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.243203    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b49dafeb-4657-4a77-bcd0-ede25c02a57c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.243359 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.243225    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.243359 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.243260    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.243359 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.243292    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b49dafeb-4657-4a77-bcd0-ede25c02a57c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.243359 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.243334    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cjvv\" (UniqueName: \"kubernetes.io/projected/b49dafeb-4657-4a77-bcd0-ede25c02a57c-kube-api-access-2cjvv\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.243627 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.243363    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.244340 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.244245    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.245915 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.245886    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b49dafeb-4657-4a77-bcd0-ede25c02a57c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.246000 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.245972    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.246082 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.246066    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b49dafeb-4657-4a77-bcd0-ede25c02a57c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.246128 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.246114    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.304331 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.304270    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b49dafeb-4657-4a77-bcd0-ede25c02a57c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.304608 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.304586    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cjvv\" (UniqueName: \"kubernetes.io/projected/b49dafeb-4657-4a77-bcd0-ede25c02a57c-kube-api-access-2cjvv\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xj658\" (UID: \"b49dafeb-4657-4a77-bcd0-ede25c02a57c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.330356 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.330331    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:17.744888 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.744858    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"]
Nov 03 16:17:17.748397 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:17:17.748371    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb49dafeb_4657_4a77_bcd0_ede25c02a57c.slice/crio-dba3cc1882e243bc7ab178237f5898c3b92e404afc36ebdeaf68afc259fabc21 WatchSource:0}: Error finding container dba3cc1882e243bc7ab178237f5898c3b92e404afc36ebdeaf68afc259fabc21: Status 404 returned error can't find the container with id dba3cc1882e243bc7ab178237f5898c3b92e404afc36ebdeaf68afc259fabc21
Nov 03 16:17:17.750322 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.750280    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:17:17.750386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:17.750362    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:17:18.566297 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:18.566254    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658" event={"ID":"b49dafeb-4657-4a77-bcd0-ede25c02a57c","Type":"ContainerStarted","Data":"eedc309615af254b91d5f2e231b725ee3166267069622fd5a13c1523c5648c27"}
Nov 03 16:17:18.566769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:18.566344    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658" event={"ID":"b49dafeb-4657-4a77-bcd0-ede25c02a57c","Type":"ContainerStarted","Data":"dba3cc1882e243bc7ab178237f5898c3b92e404afc36ebdeaf68afc259fabc21"}
Nov 03 16:17:18.566769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:18.566452    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:18.568111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:18.568075    5605 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-xj658 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:17:18.568214 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:18.568133    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658" podUID="b49dafeb-4657-4a77-bcd0-ede25c02a57c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:17:18.761683 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:18.761627    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658" podStartSLOduration=2.761614951 podStartE2EDuration="2.761614951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:17:18.761325368 +0000 UTC m=+324.585776640" watchObservedRunningTime="2026-11-03 16:17:18.761614951 +0000 UTC m=+324.586066221"
Nov 03 16:17:19.571622 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:19.571594    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xj658"
Nov 03 16:17:19.780709 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:19.780669    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"]
Nov 03 16:17:19.780970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:19.780926    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl" podUID="070e107d-c646-4b95-a962-1ad292bd3b17" containerName="discovery" containerID="cri-o://27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93" gracePeriod=30
Nov 03 16:17:20.121593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.121569    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:17:20.164247 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.164167    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-ca-configmap\") pod \"070e107d-c646-4b95-a962-1ad292bd3b17\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") "
Nov 03 16:17:20.164247 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.164216    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-kubeconfig\") pod \"070e107d-c646-4b95-a962-1ad292bd3b17\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") "
Nov 03 16:17:20.164247 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.164240    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-dns-cert\") pod \"070e107d-c646-4b95-a962-1ad292bd3b17\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") "
Nov 03 16:17:20.164451 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.164263    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-istio-token\" (UniqueName: \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-istio-token\") pod \"070e107d-c646-4b95-a962-1ad292bd3b17\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") "
Nov 03 16:17:20.164451 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.164293    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-cacerts\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-cacerts\") pod \"070e107d-c646-4b95-a962-1ad292bd3b17\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") "
Nov 03 16:17:20.164677 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.164642    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-ca-configmap" pod "070e107d-c646-4b95-a962-1ad292bd3b17" (UID: "070e107d-c646-4b95-a962-1ad292bd3b17"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:17:20.166841 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.166805    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-istio-token" pod "070e107d-c646-4b95-a962-1ad292bd3b17" (UID: "070e107d-c646-4b95-a962-1ad292bd3b17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:17:20.166841 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.166816    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-dns-cert" pod "070e107d-c646-4b95-a962-1ad292bd3b17" (UID: "070e107d-c646-4b95-a962-1ad292bd3b17"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:17:20.166985 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.166871    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-cacerts" pod "070e107d-c646-4b95-a962-1ad292bd3b17" (UID: "070e107d-c646-4b95-a962-1ad292bd3b17"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:17:20.166985 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.166895    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-kubeconfig" pod "070e107d-c646-4b95-a962-1ad292bd3b17" (UID: "070e107d-c646-4b95-a962-1ad292bd3b17"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:17:20.265423 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.265393    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/070e107d-c646-4b95-a962-1ad292bd3b17-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/070e107d-c646-4b95-a962-1ad292bd3b17-local-certs\") pod \"070e107d-c646-4b95-a962-1ad292bd3b17\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") "
Nov 03 16:17:20.265582 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.265437    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-kube-api-access-ggp89\" (UniqueName: \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-kube-api-access-ggp89\") pod \"070e107d-c646-4b95-a962-1ad292bd3b17\" (UID: \"070e107d-c646-4b95-a962-1ad292bd3b17\") "
Nov 03 16:17:20.265674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.265656    5605 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-ca-configmap\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:17:20.265714 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.265684    5605 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-kubeconfig\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:17:20.265714 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.265699    5605 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-istio-csr-dns-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:17:20.265802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.265712    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:17:20.265802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.265752    5605 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/070e107d-c646-4b95-a962-1ad292bd3b17-cacerts\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:17:20.267668 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.267636    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/070e107d-c646-4b95-a962-1ad292bd3b17-local-certs" pod "070e107d-c646-4b95-a962-1ad292bd3b17" (UID: "070e107d-c646-4b95-a962-1ad292bd3b17"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:17:20.267668 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.267654    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-kube-api-access-ggp89" pod "070e107d-c646-4b95-a962-1ad292bd3b17" (UID: "070e107d-c646-4b95-a962-1ad292bd3b17"). InnerVolumeSpecName "kube-api-access-ggp89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:17:20.366106 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.366066    5605 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/070e107d-c646-4b95-a962-1ad292bd3b17-local-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:17:20.366106 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.366097    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggp89\" (UniqueName: \"kubernetes.io/projected/070e107d-c646-4b95-a962-1ad292bd3b17-kube-api-access-ggp89\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:17:20.576458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.576374    5605 generic.go:356] "Generic (PLEG): container finished" podID="070e107d-c646-4b95-a962-1ad292bd3b17" containerID="27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93" exitCode=0
Nov 03 16:17:20.576458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.576426    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"
Nov 03 16:17:20.576936 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.576460    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl" event={"ID":"070e107d-c646-4b95-a962-1ad292bd3b17","Type":"ContainerDied","Data":"27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93"}
Nov 03 16:17:20.576936 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.576508    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl" event={"ID":"070e107d-c646-4b95-a962-1ad292bd3b17","Type":"ContainerDied","Data":"710780ada957b94055ebc2b3f551ea6669ecc9075a31e2ea633a3322a5757a53"}
Nov 03 16:17:20.576936 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.576527    5605 scope.go:122] "RemoveContainer" containerID="27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93"
Nov 03 16:17:20.584573 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.584551    5605 scope.go:122] "RemoveContainer" containerID="27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93"
Nov 03 16:17:20.584903 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:17:20.584882    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93\": container with ID starting with 27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93 not found: ID does not exist" containerID="27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93"
Nov 03 16:17:20.584952 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.584914    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93"} err="failed to get container status \"27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93\": rpc error: code = NotFound desc = could not find container \"27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93\": container with ID starting with 27ba2dd4fc64be37d90e88a737cffddab57e4f123e06853d500fcbaa71822b93 not found: ID does not exist"
Nov 03 16:17:20.735743 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.735697    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"]
Nov 03 16:17:20.782383 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:20.782356    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jznfl"]
Nov 03 16:17:22.306530 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:22.306495    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="070e107d-c646-4b95-a962-1ad292bd3b17" path="/var/lib/kubelet/pods/070e107d-c646-4b95-a962-1ad292bd3b17/volumes"
Nov 03 16:17:33.390918 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:17:33.390875    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:17:52.792591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.792553    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"]
Nov 03 16:17:52.793067 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.792861    5605 cpu_manager.go:414] "containerMap: removing container" podUID="070e107d-c646-4b95-a962-1ad292bd3b17" containerName="discovery"
Nov 03 16:17:52.793067 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.792873    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="070e107d-c646-4b95-a962-1ad292bd3b17" containerName="discovery"
Nov 03 16:17:52.793067 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.792946    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="070e107d-c646-4b95-a962-1ad292bd3b17" containerName="discovery"
Nov 03 16:17:52.795818 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.795801    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:52.892270 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.892245    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7k42d\""
Nov 03 16:17:52.892420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.892311    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 03 16:17:52.892420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.892364    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 16:17:52.893057 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.893046    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 16:17:52.893101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.893060    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 03 16:17:52.894228 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.894212    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"]
Nov 03 16:17:52.913690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.913664    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-metrics-certs\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:52.913819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.913716    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/703df78f-e3d3-4876-b5fb-72e268c21839-openshift-service-ca\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:52.913819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.913806    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpnlj\" (UniqueName: \"kubernetes.io/projected/703df78f-e3d3-4876-b5fb-72e268c21839-kube-api-access-xpnlj\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:52.913894 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:52.913836    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-maas-api-tls\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:53.014277 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.014245    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-metrics-certs\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:53.014429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.014298    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/703df78f-e3d3-4876-b5fb-72e268c21839-openshift-service-ca\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:53.014429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.014317    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpnlj\" (UniqueName: \"kubernetes.io/projected/703df78f-e3d3-4876-b5fb-72e268c21839-kube-api-access-xpnlj\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:53.014429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.014335    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-maas-api-tls\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:53.015075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.015015    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/703df78f-e3d3-4876-b5fb-72e268c21839-openshift-service-ca\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:53.016910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.016888    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-maas-api-tls\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:53.017014 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.016924    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-metrics-certs\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:53.072348 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.072287    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpnlj\" (UniqueName: \"kubernetes.io/projected/703df78f-e3d3-4876-b5fb-72e268c21839-kube-api-access-xpnlj\") pod \"maas-api-dfc545664-qnt5c\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") " pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:53.107936 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.107914    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:53.472777 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.472745    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"]
Nov 03 16:17:53.476457 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:17:53.476424    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703df78f_e3d3_4876_b5fb_72e268c21839.slice/crio-412546c9e636a663d604254504556174c2b52a65d5ec2ee4b5d836f8e75826ac WatchSource:0}: Error finding container 412546c9e636a663d604254504556174c2b52a65d5ec2ee4b5d836f8e75826ac: Status 404 returned error can't find the container with id 412546c9e636a663d604254504556174c2b52a65d5ec2ee4b5d836f8e75826ac
Nov 03 16:17:53.477864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.477846    5605 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 16:17:53.709633 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:53.709598    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c" event={"ID":"703df78f-e3d3-4876-b5fb-72e268c21839","Type":"ContainerStarted","Data":"412546c9e636a663d604254504556174c2b52a65d5ec2ee4b5d836f8e75826ac"}
Nov 03 16:17:56.725854 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:56.725759    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c" event={"ID":"703df78f-e3d3-4876-b5fb-72e268c21839","Type":"ContainerStarted","Data":"250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c"}
Nov 03 16:17:56.726219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:56.725880    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:17:56.841797 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:17:56.841748    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c" podStartSLOduration=1.914229657 podStartE2EDuration="4.841705353s" totalImagesPullingTime="2.927475696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:17:56.840882383 +0000 UTC m=+362.665333655" watchObservedRunningTime="2026-11-03 16:17:56.841705353 +0000 UTC m=+362.666156606"
Nov 03 16:18:02.734948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:18:02.734918    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 16:18:03.392349 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:18:03.392313    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:18:33.391157 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:18:33.391074    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:19:03.391671 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:19:03.391635    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:19:33.390162 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:19:33.390122    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:19:37.449407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.449365    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"]
Nov 03 16:19:37.452302 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.452286    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.512502 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.512465    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 03 16:19:37.548342 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.548308    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fh4dr\""
Nov 03 16:19:37.548342 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.548335    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 03 16:19:37.548573 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.548452    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Nov 03 16:19:37.549577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.549553    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"]
Nov 03 16:19:37.563064 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.563037    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/983fedf8-220b-4e70-bf19-1e72aa4391e5-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.563154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.563090    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.563208 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.563163    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.563282 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.563263    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9w2d\" (UniqueName: \"kubernetes.io/projected/983fedf8-220b-4e70-bf19-1e72aa4391e5-kube-api-access-h9w2d\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.563339 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.563321    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.563403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.563350    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.664411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.664373    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.664411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.664408    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.664649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.664438    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9w2d\" (UniqueName: \"kubernetes.io/projected/983fedf8-220b-4e70-bf19-1e72aa4391e5-kube-api-access-h9w2d\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.664649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.664468    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.664649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.664483    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.664649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.664504    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/983fedf8-220b-4e70-bf19-1e72aa4391e5-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.664893 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.664861    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.664939 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.664898    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.664976 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.664962    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.667041 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.667023    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/983fedf8-220b-4e70-bf19-1e72aa4391e5-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.667160 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.667144    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/983fedf8-220b-4e70-bf19-1e72aa4391e5-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.723387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.723325    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9w2d\" (UniqueName: \"kubernetes.io/projected/983fedf8-220b-4e70-bf19-1e72aa4391e5-kube-api-access-h9w2d\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mf979\" (UID: \"983fedf8-220b-4e70-bf19-1e72aa4391e5\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:37.765272 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:37.765236    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:38.136357 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:38.136324    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"]
Nov 03 16:19:38.139541 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:19:38.139508    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983fedf8_220b_4e70_bf19_1e72aa4391e5.slice/crio-d8534e507ee2e64cde16addc4426a280d0c14bc0e65dabd5a2fd9b393eeeb090 WatchSource:0}: Error finding container d8534e507ee2e64cde16addc4426a280d0c14bc0e65dabd5a2fd9b393eeeb090: Status 404 returned error can't find the container with id d8534e507ee2e64cde16addc4426a280d0c14bc0e65dabd5a2fd9b393eeeb090
Nov 03 16:19:39.125005 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:39.124962    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979" event={"ID":"983fedf8-220b-4e70-bf19-1e72aa4391e5","Type":"ContainerStarted","Data":"d8534e507ee2e64cde16addc4426a280d0c14bc0e65dabd5a2fd9b393eeeb090"}
Nov 03 16:19:40.129914 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:40.129879    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979" event={"ID":"983fedf8-220b-4e70-bf19-1e72aa4391e5","Type":"ContainerStarted","Data":"b54a2b795cc34fe2e1b92cc8c74b9f7b02050f5ffe634f3d773ed16b84b029ee"}
Nov 03 16:19:40.243144 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:40.243094    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979" podStartSLOduration=1.662022133 podStartE2EDuration="3.243074322s" totalImagesPullingTime="1.581052189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:19:40.242388168 +0000 UTC m=+466.066839439" watchObservedRunningTime="2026-11-03 16:19:40.243074322 +0000 UTC m=+466.067525590"
Nov 03 16:19:40.765961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:40.765927    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:47.765919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:47.765887    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:47.778231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:47.778209    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:48.171301 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.171269    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mf979"
Nov 03 16:19:48.307399 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.307361    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-4jdmt"]
Nov 03 16:19:48.310474 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.310453    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.349741 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.349695    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.349885 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.349765    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.349932 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.349893    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-home\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.349990 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.349935    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.350033 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.350007    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843c64aa-38ea-496a-bd0c-13158d516c7c-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.350077 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.350037    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2tpv\" (UniqueName: \"kubernetes.io/projected/843c64aa-38ea-496a-bd0c-13158d516c7c-kube-api-access-x2tpv\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.364093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.364070    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Nov 03 16:19:48.364631 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.364605    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-4jdmt"]
Nov 03 16:19:48.450979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.450893    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.450979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.450937    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.451219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.451011    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-home\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.451219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.451056    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.451219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.451073    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843c64aa-38ea-496a-bd0c-13158d516c7c-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.451219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.451092    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2tpv\" (UniqueName: \"kubernetes.io/projected/843c64aa-38ea-496a-bd0c-13158d516c7c-kube-api-access-x2tpv\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.451505 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.451485    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.451580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.451532    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.451707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.451683    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-home\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.453880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.453855    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/843c64aa-38ea-496a-bd0c-13158d516c7c-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.453880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.453878    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/843c64aa-38ea-496a-bd0c-13158d516c7c-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.507248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.507221    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2tpv\" (UniqueName: \"kubernetes.io/projected/843c64aa-38ea-496a-bd0c-13158d516c7c-kube-api-access-x2tpv\") pod \"echo-simulator-kserve-64b6f8f96c-4jdmt\" (UID: \"843c64aa-38ea-496a-bd0c-13158d516c7c\") " pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.622534 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.622493    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:48.988826 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:48.988794    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-4jdmt"]
Nov 03 16:19:48.991613 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:19:48.991584    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843c64aa_38ea_496a_bd0c_13158d516c7c.slice/crio-2e943c9976d22d7c9bda43cab9144ad6f2d62b845667a21a6394a393cad50f29 WatchSource:0}: Error finding container 2e943c9976d22d7c9bda43cab9144ad6f2d62b845667a21a6394a393cad50f29: Status 404 returned error can't find the container with id 2e943c9976d22d7c9bda43cab9144ad6f2d62b845667a21a6394a393cad50f29
Nov 03 16:19:49.165260 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:49.165222    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt" event={"ID":"843c64aa-38ea-496a-bd0c-13158d516c7c","Type":"ContainerStarted","Data":"2e943c9976d22d7c9bda43cab9144ad6f2d62b845667a21a6394a393cad50f29"}
Nov 03 16:19:52.177697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:52.177656    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt" event={"ID":"843c64aa-38ea-496a-bd0c-13158d516c7c","Type":"ContainerStarted","Data":"10a58f81d2ceefc8bf228d84bb3b584c85f855e01098c7804ce5213f4768c0a3"}
Nov 03 16:19:52.297718 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:52.297648    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt" podStartSLOduration=1.542533831 podStartE2EDuration="4.297630051s" totalImagesPullingTime="2.75509622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:19:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:19:52.297246578 +0000 UTC m=+478.121697863" watchObservedRunningTime="2026-11-03 16:19:52.297630051 +0000 UTC m=+478.122081323"
Nov 03 16:19:52.622870 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:52.622838    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:58.033326 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.033289    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"]
Nov 03 16:19:58.036667 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.036647    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.115136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.115108    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 03 16:19:58.132530 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.132500    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x5cv\" (UniqueName: \"kubernetes.io/projected/f237b2f1-9a28-419e-805a-0d77cbda7230-kube-api-access-6x5cv\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.132645 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.132546    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.132645 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.132627    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.132741 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.132654    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.132741 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.132675    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f237b2f1-9a28-419e-805a-0d77cbda7230-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.132807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.132742    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.134197 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.134174    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"]
Nov 03 16:19:58.234049 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.234019    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x5cv\" (UniqueName: \"kubernetes.io/projected/f237b2f1-9a28-419e-805a-0d77cbda7230-kube-api-access-6x5cv\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.234049 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.234060    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.234253 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.234173    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.234253 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.234216    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.234253 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.234247    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f237b2f1-9a28-419e-805a-0d77cbda7230-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.234423 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.234269    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.234478 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.234442    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.234531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.234479    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.234604 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.234584    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.236747 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.236693    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f237b2f1-9a28-419e-805a-0d77cbda7230-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.236862 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.236844    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f237b2f1-9a28-419e-805a-0d77cbda7230-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.291460 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.291393    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x5cv\" (UniqueName: \"kubernetes.io/projected/f237b2f1-9a28-419e-805a-0d77cbda7230-kube-api-access-6x5cv\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld\" (UID: \"f237b2f1-9a28-419e-805a-0d77cbda7230\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.362467 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.362435    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:19:58.623340 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.623303    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:58.635710 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.635687    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:19:58.730063 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:58.730034    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"]
Nov 03 16:19:58.733178 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:19:58.733155    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf237b2f1_9a28_419e_805a_0d77cbda7230.slice/crio-aae46767c0dff2eb635b4d3cceb229c9fdaa2f1586250068c3872cc4f414b58c WatchSource:0}: Error finding container aae46767c0dff2eb635b4d3cceb229c9fdaa2f1586250068c3872cc4f414b58c: Status 404 returned error can't find the container with id aae46767c0dff2eb635b4d3cceb229c9fdaa2f1586250068c3872cc4f414b58c
Nov 03 16:19:59.206168 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:59.206130    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" event={"ID":"f237b2f1-9a28-419e-805a-0d77cbda7230","Type":"ContainerStarted","Data":"aae46767c0dff2eb635b4d3cceb229c9fdaa2f1586250068c3872cc4f414b58c"}
Nov 03 16:19:59.217880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:19:59.217848    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-4jdmt"
Nov 03 16:20:00.212852 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:00.212819    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" event={"ID":"f237b2f1-9a28-419e-805a-0d77cbda7230","Type":"ContainerStarted","Data":"d2f912f1a891fe2fa3f74467791aeccea7897f9ccf448154851f65c9bde9c79b"}
Nov 03 16:20:00.325556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:00.325502    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podStartSLOduration=2.754414699 podStartE2EDuration="3.325483362s" totalImagesPullingTime="571.068663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:20:00.324429702 +0000 UTC m=+486.148880992" watchObservedRunningTime="2026-11-03 16:20:00.325483362 +0000 UTC m=+486.149934637"
Nov 03 16:20:00.363137 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:00.363108    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:20:03.394320 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:20:03.392250    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:20:05.536690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.536656    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"]
Nov 03 16:20:05.540086 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.540068    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.597559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.597524    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2qzb\" (UniqueName: \"kubernetes.io/projected/d5cc21a5-ffff-470e-9f8e-41acc80f2132-kube-api-access-v2qzb\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.597838 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.597570    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5cc21a5-ffff-470e-9f8e-41acc80f2132-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.597937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.597885    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.597937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.597912    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.598057 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.597946    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.598057 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.597979    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.636802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.636773    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Nov 03 16:20:05.637879 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.637859    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"]
Nov 03 16:20:05.699283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.699245    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.699283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.699281    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.699536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.699312    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.699536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.699439    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.699536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.699478    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2qzb\" (UniqueName: \"kubernetes.io/projected/d5cc21a5-ffff-470e-9f8e-41acc80f2132-kube-api-access-v2qzb\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.699536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.699518    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5cc21a5-ffff-470e-9f8e-41acc80f2132-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.699790 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.699768    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.699868 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.699813    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.699936 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.699904    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.701838 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.701810    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5cc21a5-ffff-470e-9f8e-41acc80f2132-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.702070 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.702053    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5cc21a5-ffff-470e-9f8e-41acc80f2132-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.756321 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.756291    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2qzb\" (UniqueName: \"kubernetes.io/projected/d5cc21a5-ffff-470e-9f8e-41acc80f2132-kube-api-access-v2qzb\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-t7rp9\" (UID: \"d5cc21a5-ffff-470e-9f8e-41acc80f2132\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:05.852023 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:05.851991    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:06.216782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:06.216543    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"]
Nov 03 16:20:06.219421 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:20:06.219381    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cc21a5_ffff_470e_9f8e_41acc80f2132.slice/crio-6349aec79a13f915910884883c3bc6e3595da75231cdc68eec61c7e55691bf53 WatchSource:0}: Error finding container 6349aec79a13f915910884883c3bc6e3595da75231cdc68eec61c7e55691bf53: Status 404 returned error can't find the container with id 6349aec79a13f915910884883c3bc6e3595da75231cdc68eec61c7e55691bf53
Nov 03 16:20:06.237406 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:06.237378    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9" event={"ID":"d5cc21a5-ffff-470e-9f8e-41acc80f2132","Type":"ContainerStarted","Data":"6349aec79a13f915910884883c3bc6e3595da75231cdc68eec61c7e55691bf53"}
Nov 03 16:20:07.242516 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:07.242421    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9" event={"ID":"d5cc21a5-ffff-470e-9f8e-41acc80f2132","Type":"ContainerStarted","Data":"4ebd62c9e2327f8a5bfef48c2237a3ba6095ac978d76da631ec5054dda83d06a"}
Nov 03 16:20:07.354939 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:07.354888    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9" podStartSLOduration=1.709046388 podStartE2EDuration="2.35487209s" totalImagesPullingTime="645.825702ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:20:07.353686773 +0000 UTC m=+493.178138034" watchObservedRunningTime="2026-11-03 16:20:07.35487209 +0000 UTC m=+493.179323360"
Nov 03 16:20:07.852659 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:07.852606    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:08.363184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:08.363152    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:20:08.375516 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:08.375491    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:20:09.260930 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:09.260902    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:20:15.852793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.852757    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:15.865111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.865089    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:15.866840 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.866815    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"]
Nov 03 16:20:15.870286 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.870265    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:15.926689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.926665    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mw47n\""
Nov 03 16:20:15.964532 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.964494    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"]
Nov 03 16:20:15.990532 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.990492    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:15.990696 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.990643    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:15.990779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.990761    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03f9ac99-4327-4059-942b-f1ce28a36b72-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:15.990827 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.990790    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:15.990827 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.990819    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:15.990982 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.990961    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:15.991059 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.991040    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:15.991117 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.991069    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4djxb\" (UniqueName: \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-kube-api-access-4djxb\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:15.991152 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:15.991133    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.091620 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.091586    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03f9ac99-4327-4059-942b-f1ce28a36b72-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.091620 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.091627    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.091948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.091653    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.091948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.091702    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.091948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.091775    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.091948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.091797    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4djxb\" (UniqueName: \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-kube-api-access-4djxb\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.091948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.091838    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.091948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.091877    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.091948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.091933    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.092357 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.092152    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.092514 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.092490    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.092812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.092776    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.093091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.092980    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.093388 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.093364    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03f9ac99-4327-4059-942b-f1ce28a36b72-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.095371 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.095350    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.095510 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.095490    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.148349 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.148282    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.148481 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.148465    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4djxb\" (UniqueName: \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-kube-api-access-4djxb\") pod \"maas-default-gateway-openshift-default-5bc884dd87-sl8t8\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.181320 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.181295    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:16.290741 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.290699    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-t7rp9"
Nov 03 16:20:16.612098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:16.612058    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"]
Nov 03 16:20:16.615343 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:20:16.615310    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f9ac99_4327_4059_942b_f1ce28a36b72.slice/crio-6262d5500f0d834e31aa58128dbb2b378b7d015ee285667d49c30242b5cdd023 WatchSource:0}: Error finding container 6262d5500f0d834e31aa58128dbb2b378b7d015ee285667d49c30242b5cdd023: Status 404 returned error can't find the container with id 6262d5500f0d834e31aa58128dbb2b378b7d015ee285667d49c30242b5cdd023
Nov 03 16:20:17.286014 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:17.285978    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" event={"ID":"03f9ac99-4327-4059-942b-f1ce28a36b72","Type":"ContainerStarted","Data":"6262d5500f0d834e31aa58128dbb2b378b7d015ee285667d49c30242b5cdd023"}
Nov 03 16:20:22.545684 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:22.545648    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:20:22.545986 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:22.545718    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:20:22.545986 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:22.545771    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:20:23.314134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:23.314094    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" event={"ID":"03f9ac99-4327-4059-942b-f1ce28a36b72","Type":"ContainerStarted","Data":"8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0"}
Nov 03 16:20:23.479459 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:23.479397    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" podStartSLOduration=2.552133488 podStartE2EDuration="8.479380692s" totalImagesPullingTime="5.927247204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:20:23.478190029 +0000 UTC m=+509.302641300" watchObservedRunningTime="2026-11-03 16:20:23.479380692 +0000 UTC m=+509.303831964"
Nov 03 16:20:24.201751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:24.198432    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-sl8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 16:20:24.201751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:24.198508    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 16:20:24.201751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:24.198805    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:25.191775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:25.189018    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-sl8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 16:20:25.191775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:25.189077    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 16:20:26.199763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:26.199648    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-sl8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 16:20:26.199763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:26.199708    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 16:20:27.183111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:27.183073    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-sl8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 16:20:27.183392 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:27.183363    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 16:20:28.189765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:28.189089    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-sl8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 16:20:28.189765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:28.189150    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 16:20:29.181871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:29.181839    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-sl8t8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 16:20:29.182084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:29.181894    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 16:20:30.789639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:30.789591    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:30.790159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:30.790073    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:30.790857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:30.790835    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:20:31.008284 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.008255    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"]
Nov 03 16:20:31.011557 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.011525    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.048467 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.048396    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.048467 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.048429    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxqt5\" (UniqueName: \"kubernetes.io/projected/b0246a86-f897-42f0-88eb-38c00c8b1bc3-kube-api-access-dxqt5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.048467 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.048450    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.048671 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.048474    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.048671 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.048593    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.048671 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.048630    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.048671 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.048659    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.048837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.048710    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.048837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.048813    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.073017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.072992    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"]
Nov 03 16:20:31.149345 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149313    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.149345 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149344    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxqt5\" (UniqueName: \"kubernetes.io/projected/b0246a86-f897-42f0-88eb-38c00c8b1bc3-kube-api-access-dxqt5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.149596 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149368    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.149596 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149398    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.149596 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149422    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.149596 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149443    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.149596 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149484    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.149882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149741    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.149882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149832    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.149882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149871    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.150033 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.149895    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.150091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.150029    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.150303 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.150279    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.150421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.150370    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.152231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.152209    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.152428 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.152410    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.208232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.208201    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0246a86-f897-42f0-88eb-38c00c8b1bc3-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.208953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.208928    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxqt5\" (UniqueName: \"kubernetes.io/projected/b0246a86-f897-42f0-88eb-38c00c8b1bc3-kube-api-access-dxqt5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rn227\" (UID: \"b0246a86-f897-42f0-88eb-38c00c8b1bc3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.323976 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.323888    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:31.748281 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.748253    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"]
Nov 03 16:20:31.751233 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:20:31.751200    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0246a86_f897_42f0_88eb_38c00c8b1bc3.slice/crio-57f6160a486bd19c86601d59515e8e969ae3769fef6dda875b041f1a98447e3a WatchSource:0}: Error finding container 57f6160a486bd19c86601d59515e8e969ae3769fef6dda875b041f1a98447e3a: Status 404 returned error can't find the container with id 57f6160a486bd19c86601d59515e8e969ae3769fef6dda875b041f1a98447e3a
Nov 03 16:20:31.753399 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.753366    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:20:31.753515 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.753446    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:20:31.753515 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:31.753493    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:20:32.351166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:32.351128    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" event={"ID":"b0246a86-f897-42f0-88eb-38c00c8b1bc3","Type":"ContainerStarted","Data":"f2060f069be4d8400026f55cb36b287d54c70a255b9bde05e29db997944e2a46"}
Nov 03 16:20:32.351595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:32.351185    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" event={"ID":"b0246a86-f897-42f0-88eb-38c00c8b1bc3","Type":"ContainerStarted","Data":"57f6160a486bd19c86601d59515e8e969ae3769fef6dda875b041f1a98447e3a"}
Nov 03 16:20:32.513404 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:32.513351    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" podStartSLOduration=2.513335273 podStartE2EDuration="2.513335273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:20:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:20:32.512373056 +0000 UTC m=+518.336824329" watchObservedRunningTime="2026-11-03 16:20:32.513335273 +0000 UTC m=+518.337786543"
Nov 03 16:20:33.324452 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:33.324418    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:33.325690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:33.325667    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rn227 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 16:20:33.325811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:33.325720    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" podUID="b0246a86-f897-42f0-88eb-38c00c8b1bc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 16:20:33.410004 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:20:33.409961    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:20:34.324574 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:34.324541    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rn227 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 16:20:34.324816 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:34.324608    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" podUID="b0246a86-f897-42f0-88eb-38c00c8b1bc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 16:20:35.324667 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:35.324628    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rn227 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 16:20:35.325105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:35.324686    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" podUID="b0246a86-f897-42f0-88eb-38c00c8b1bc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 16:20:36.324586 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:36.324548    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rn227 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 16:20:36.324793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:36.324599    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" podUID="b0246a86-f897-42f0-88eb-38c00c8b1bc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 16:20:37.324497 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:37.324452    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rn227 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 16:20:37.324783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:37.324508    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" podUID="b0246a86-f897-42f0-88eb-38c00c8b1bc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 16:20:38.325020 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:38.324990    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rn227 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 16:20:38.325508 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:38.325050    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" podUID="b0246a86-f897-42f0-88eb-38c00c8b1bc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 16:20:40.080019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:40.079984    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:40.080466 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:40.080421    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:20:40.081406 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:20:40.081386    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227"
Nov 03 16:21:03.390165 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:21:03.390125    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:21:33.390496 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:21:33.390416    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:21:54.358678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.358650    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:21:54.361418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.361398    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:21:54.553217 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.553180    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"]
Nov 03 16:21:54.556496 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.556474    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.607602 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.607570    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-42da0f-openshift-default-dockercfg-l8c4p\""
Nov 03 16:21:54.621519 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.621426    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-envoy\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.621519 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.621471    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-credential-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.621706 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.621564    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.621706 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.621599    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-token\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.621706 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.621653    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istiod-ca-cert\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.621706 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.621677    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-podinfo\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.621876 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.621805    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-data\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.621919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.621907    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-certs\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.621973 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.621955    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59h5v\" (UniqueName: \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-kube-api-access-59h5v\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.655643 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.655608    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"]
Nov 03 16:21:54.723174 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723138    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-certs\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723174 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723175    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59h5v\" (UniqueName: \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-kube-api-access-59h5v\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723222    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-envoy\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723254    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-credential-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723296    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723322    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-token\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723349    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istiod-ca-cert\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723378    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-podinfo\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723684 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723433    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-data\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723684 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723645    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-certs\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723814 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723678    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-credential-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723814 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723781    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.723814 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.723801    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-data\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.724130 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.724107    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istiod-ca-cert\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.725636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.725617    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-envoy\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.726147 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.726119    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-podinfo\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.780452 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.780422    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-token\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.780591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.780520    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59h5v\" (UniqueName: \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-kube-api-access-59h5v\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.867378 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.867340    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"]
Nov 03 16:21:54.869096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.869070    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:54.871853 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.871798    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:54.923252 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.923052    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-6ae690-openshift-default-dockercfg-6s4h2\""
Nov 03 16:21:54.926416 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.926383    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:54.926535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.926448    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjcl8\" (UniqueName: \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-kube-api-access-sjcl8\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:54.926535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.926494    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:54.926644 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.926532    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:54.926644 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.926586    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:54.926644 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.926617    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:54.926812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.926649    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:54.926812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.926683    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:54.926920 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.926826    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:54.931809 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:54.931739    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"]
Nov 03 16:21:55.028277 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028244    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjcl8\" (UniqueName: \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-kube-api-access-sjcl8\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.028472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028287    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.028472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028316    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.028472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028362    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.028472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028386    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.028472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028417    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.028472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028450    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.028849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028496    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.028849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028539    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.028849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028768    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.029032 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028890    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.029032 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.028925    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.029032 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.029024    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.029222 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.029199    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.030721 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.030699    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.031073 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.031055    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.086180 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.086146    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjcl8\" (UniqueName: \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-kube-api-access-sjcl8\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.086683 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.086660    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.192973 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.192879    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:55.205618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.205597    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"]
Nov 03 16:21:55.209177 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.209155    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.231050 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.230991    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.231050 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.231045    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.231254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.231073    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-487zc\" (UniqueName: \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-kube-api-access-487zc\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.231254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.231228    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.231403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.231385    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.231476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.231418    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.231528 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.231497    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.231568 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.231543    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.231616 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.231601    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.265788 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.263188    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-2bb620-openshift-default-dockercfg-m62nq\""
Nov 03 16:21:55.270519 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.270490    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"]
Nov 03 16:21:55.301694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.301667    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"]
Nov 03 16:21:55.308774 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:21:55.308746    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd53f66_eba2_4ad7_a908_a42c9b6914d9.slice/crio-32b8f4df1bd59c9e2e8dcff79f851ffe834a232d855f93433791b8bd0732a266 WatchSource:0}: Error finding container 32b8f4df1bd59c9e2e8dcff79f851ffe834a232d855f93433791b8bd0732a266: Status 404 returned error can't find the container with id 32b8f4df1bd59c9e2e8dcff79f851ffe834a232d855f93433791b8bd0732a266
Nov 03 16:21:55.310937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.310906    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:55.311017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.310971    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:55.311017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.311007    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:55.332372 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.332348    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.332488 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.332418    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.332488 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.332445    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.332488 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.332470    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-487zc\" (UniqueName: \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-kube-api-access-487zc\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.332647 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.332498    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.332647 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.332560    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.332647 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.332588    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.332647 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.332629    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.332842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.332659    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.333016 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.332992    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.333203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.333179    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.333271 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.333205    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.333271 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.333223    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.333345 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.333186    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.334764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.334721    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.334863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.334845    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.388828 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.388801    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.389966 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.389942    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-487zc\" (UniqueName: \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-kube-api-access-487zc\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.522975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.522882    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:55.579398 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.579368    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"]
Nov 03 16:21:55.587844 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:21:55.587797    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod806a66bb_48eb_41ae_8fbe_c7869b6040f5.slice/crio-65b79546d2534b60c153954d105b1b4f8f5ce567f6c6b80a5eadc3bd9189362b WatchSource:0}: Error finding container 65b79546d2534b60c153954d105b1b4f8f5ce567f6c6b80a5eadc3bd9189362b: Status 404 returned error can't find the container with id 65b79546d2534b60c153954d105b1b4f8f5ce567f6c6b80a5eadc3bd9189362b
Nov 03 16:21:55.590963 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.590924    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:55.591095 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.591072    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:55.591162 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.591127    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:55.708709 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.707818    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" event={"ID":"806a66bb-48eb-41ae-8fbe-c7869b6040f5","Type":"ContainerStarted","Data":"5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d"}
Nov 03 16:21:55.708709 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.707874    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" event={"ID":"806a66bb-48eb-41ae-8fbe-c7869b6040f5","Type":"ContainerStarted","Data":"65b79546d2534b60c153954d105b1b4f8f5ce567f6c6b80a5eadc3bd9189362b"}
Nov 03 16:21:55.712392 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.710552    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" event={"ID":"bfd53f66-eba2-4ad7-a908-a42c9b6914d9","Type":"ContainerStarted","Data":"e2c9c2e15e45c9815a9b258c622cba8c360020710b63b1a319dec8d5e4498a93"}
Nov 03 16:21:55.712392 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.710597    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" event={"ID":"bfd53f66-eba2-4ad7-a908-a42c9b6914d9","Type":"ContainerStarted","Data":"32b8f4df1bd59c9e2e8dcff79f851ffe834a232d855f93433791b8bd0732a266"}
Nov 03 16:21:55.825484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.825376    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podStartSLOduration=1.8253620750000001 podStartE2EDuration="1.825362075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:21:55.824094635 +0000 UTC m=+601.648545906" watchObservedRunningTime="2026-11-03 16:21:55.825362075 +0000 UTC m=+601.649813346"
Nov 03 16:21:55.867196 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.867164    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"]
Nov 03 16:21:55.870939 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.870918    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:55.871060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.871011    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:55.922954 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.922923    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-c88d54-openshift-default-dockercfg-7w7bk\""
Nov 03 16:21:55.925482 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.925454    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"]
Nov 03 16:21:55.934764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.929193    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"]
Nov 03 16:21:55.934764 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:21:55.934657    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6ffa84_a0a2_4fc5_ac4a_ceff5a1d2066.slice/crio-ea116b30c0a183a50b518252e289e6a69a71c00aa05e35ca27e854e81f9a4f3e WatchSource:0}: Error finding container ea116b30c0a183a50b518252e289e6a69a71c00aa05e35ca27e854e81f9a4f3e: Status 404 returned error can't find the container with id ea116b30c0a183a50b518252e289e6a69a71c00aa05e35ca27e854e81f9a4f3e
Nov 03 16:21:55.937609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.937571    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:55.937697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.937656    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:55.937782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.937702    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:55.938556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.938530    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-data\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:55.938658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.938609    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:55.938739 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.938678    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-certs\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:55.938739 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.938705    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-podinfo\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:55.938859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.938778    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2765h\" (UniqueName: \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-kube-api-access-2765h\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:55.938859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.938820    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-token\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:55.938859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.938841    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae2b9d64-981e-419f-84be-3beae332c9d1-istiod-ca-cert\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:55.939018 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.938871    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-credential-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:55.939018 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.938888    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-envoy\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:55.941677 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:55.941622    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" podStartSLOduration=1.941607983 podStartE2EDuration="1.941607983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:21:55.940518645 +0000 UTC m=+601.764969934" watchObservedRunningTime="2026-11-03 16:21:55.941607983 +0000 UTC m=+601.766059261"
Nov 03 16:21:56.040240 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040209    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-token\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.040400 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040246    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae2b9d64-981e-419f-84be-3beae332c9d1-istiod-ca-cert\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.040400 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040289    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-credential-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.040400 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040317    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-envoy\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.040400 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040383    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-data\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.040612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040453    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.040612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040485    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-certs\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.040612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040509    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-podinfo\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.040612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040534    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2765h\" (UniqueName: \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-kube-api-access-2765h\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.040819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040772    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-credential-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.040943 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.040912    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.041102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.041063    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-certs\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.041169 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.041097    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae2b9d64-981e-419f-84be-3beae332c9d1-istiod-ca-cert\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.041282 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.041111    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-data\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.042911 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.042863    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-envoy\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.044313 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.043976    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-podinfo\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.096421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.096389    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-token\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.096809 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.096785    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2765h\" (UniqueName: \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-kube-api-access-2765h\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.185072 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.185039    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:56.190601 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.190572    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"]
Nov 03 16:21:56.194097 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.194076    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:56.194211 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.194154    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.199526 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.199503    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:56.245437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.244829    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.245437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.244921    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.245437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.244947    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.245437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.244976    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.245437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.245046    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.245437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.245077    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl9t4\" (UniqueName: \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-kube-api-access-sl9t4\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.245437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.245172    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.245437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.245194    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.245437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.245226    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.246068 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.245672    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-bcd756-openshift-default-dockercfg-8drrh\""
Nov 03 16:21:56.262285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.262190    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"]
Nov 03 16:21:56.347185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.347097    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.347185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.347148    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.347185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.347175    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.347455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.347219    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.347455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.347305    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.347455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.347328    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.347455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.347361    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.347455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.347441    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.347694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.347472    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl9t4\" (UniqueName: \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-kube-api-access-sl9t4\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.350773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.350618    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.354495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.353950    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.354495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.354152    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.354495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.354453    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.355940 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.355518    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.362255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.362207    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.367129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.367072    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.406109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.406077    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl9t4\" (UniqueName: \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-kube-api-access-sl9t4\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.406399 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.406130    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.509538 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.509505    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:56.608886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.608821    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"]
Nov 03 16:21:56.615387 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:21:56.615355    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2b9d64_981e_419f_84be_3beae332c9d1.slice/crio-e3f73ae9cc169a4594331b039b3780410ce10753f8e1a136097032f5f47fd6d1 WatchSource:0}: Error finding container e3f73ae9cc169a4594331b039b3780410ce10753f8e1a136097032f5f47fd6d1: Status 404 returned error can't find the container with id e3f73ae9cc169a4594331b039b3780410ce10753f8e1a136097032f5f47fd6d1
Nov 03 16:21:56.617619 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.617588    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:56.617699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.617651    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:56.617699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.617679    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:56.721752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.720769    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" event={"ID":"ae2b9d64-981e-419f-84be-3beae332c9d1","Type":"ContainerStarted","Data":"6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1"}
Nov 03 16:21:56.721752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.720858    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" event={"ID":"ae2b9d64-981e-419f-84be-3beae332c9d1","Type":"ContainerStarted","Data":"e3f73ae9cc169a4594331b039b3780410ce10753f8e1a136097032f5f47fd6d1"}
Nov 03 16:21:56.726762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.723829    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" event={"ID":"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066","Type":"ContainerStarted","Data":"975a84732baa998eb1d16061be20b77a36f077f30bf0ef318db3e284981057af"}
Nov 03 16:21:56.726762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.723897    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" event={"ID":"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066","Type":"ContainerStarted","Data":"ea116b30c0a183a50b518252e289e6a69a71c00aa05e35ca27e854e81f9a4f3e"}
Nov 03 16:21:56.726762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.723937    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:56.726762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.725370    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:21:56.837813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.837759    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" podStartSLOduration=1.837717555 podStartE2EDuration="1.837717555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:21:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:21:56.836709448 +0000 UTC m=+602.661160729" watchObservedRunningTime="2026-11-03 16:21:56.837717555 +0000 UTC m=+602.662168826"
Nov 03 16:21:56.875496 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.875422    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:56.898639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.898610    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"]
Nov 03 16:21:56.906456 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:21:56.906421    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6940c25a_c0b2_4502_bdcf_1f8b32c0fd3a.slice/crio-fe5678e72cf8cb1c2373c3e3861caa0eed405b2a91e18581d0d68540b069049c WatchSource:0}: Error finding container fe5678e72cf8cb1c2373c3e3861caa0eed405b2a91e18581d0d68540b069049c: Status 404 returned error can't find the container with id fe5678e72cf8cb1c2373c3e3861caa0eed405b2a91e18581d0d68540b069049c
Nov 03 16:21:56.909264 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.909223    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:56.909363 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.909312    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:56.909363 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:56.909354    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:21:57.076546 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.076496    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" podStartSLOduration=2.076480587 podStartE2EDuration="2.076480587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:21:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:21:57.074461731 +0000 UTC m=+602.898913001" watchObservedRunningTime="2026-11-03 16:21:57.076480587 +0000 UTC m=+602.900931858"
Nov 03 16:21:57.185697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.185608    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:57.208285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.208036    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:57.523882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.523797    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:57.528955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.528928    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:57.729554 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.729515    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" event={"ID":"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a","Type":"ContainerStarted","Data":"8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5"}
Nov 03 16:21:57.729799 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.729583    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" event={"ID":"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a","Type":"ContainerStarted","Data":"fe5678e72cf8cb1c2373c3e3861caa0eed405b2a91e18581d0d68540b069049c"}
Nov 03 16:21:57.730305 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.730278    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:57.730436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.730331    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:57.730436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.730355    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:57.731041 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.731019    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:21:57.731139 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.731072    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:21:57.731139 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.731103    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:21:57.845347 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:57.845295    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" podStartSLOduration=1.845282208 podStartE2EDuration="1.845282208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:21:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:21:57.844066034 +0000 UTC m=+603.668517305" watchObservedRunningTime="2026-11-03 16:21:57.845282208 +0000 UTC m=+603.669733547"
Nov 03 16:21:58.509715 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:58.509683    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:58.514705 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:58.514682    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:58.734549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:58.734521    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:21:58.735373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:21:58.735353    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:22:03.392463 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:22:03.392430    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:22:21.021505 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.021462    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"]
Nov 03 16:22:21.031226 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.031193    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.085035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.085003    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-2bb620-serving-cert\""
Nov 03 16:22:21.137751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.137223    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"]
Nov 03 16:22:21.148942 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.148903    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c2b816-5bf9-40f9-98d4-e325d16a869c-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.149158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.149139    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.149292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.149280    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-metrics-certs\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.149436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.149424    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpp5v\" (UniqueName: \"kubernetes.io/projected/24c2b816-5bf9-40f9-98d4-e325d16a869c-kube-api-access-kpp5v\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.250863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.250823    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.251067 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.250897    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-metrics-certs\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.251067 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.250964    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpp5v\" (UniqueName: \"kubernetes.io/projected/24c2b816-5bf9-40f9-98d4-e325d16a869c-kube-api-access-kpp5v\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.251067 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.251051    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c2b816-5bf9-40f9-98d4-e325d16a869c-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.251437 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:22:21.251413    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-serving-cert: secret "maas-api-e2e-worker-w4-2bb620-serving-cert" not found
Nov 03 16:22:21.251828 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.251802    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c2b816-5bf9-40f9-98d4-e325d16a869c-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.251936 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:22:21.251903    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-maas-api-tls podName:24c2b816-5bf9-40f9-98d4-e325d16a869c nodeName:}" failed. No retries permitted until 2026-11-03 16:22:21.751631432 +0000 UTC m=+627.576082700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-maas-api-tls") pod "maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm" (UID: "24c2b816-5bf9-40f9-98d4-e325d16a869c") : secret "maas-api-e2e-worker-w4-2bb620-serving-cert" not found
Nov 03 16:22:21.254200 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.254181    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-metrics-certs\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.346505 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.346428    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpp5v\" (UniqueName: \"kubernetes.io/projected/24c2b816-5bf9-40f9-98d4-e325d16a869c-kube-api-access-kpp5v\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.756619 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.756525    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.759147 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.759111    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:21.944648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:21.944609    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:22.375825 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:22.375788    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"]
Nov 03 16:22:22.427785 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:22:22.427717    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c2b816_5bf9_40f9_98d4_e325d16a869c.slice/crio-98e68074a3e7543a8b8de3904782001b5da49106deff7183548af1dced917ae3 WatchSource:0}: Error finding container 98e68074a3e7543a8b8de3904782001b5da49106deff7183548af1dced917ae3: Status 404 returned error can't find the container with id 98e68074a3e7543a8b8de3904782001b5da49106deff7183548af1dced917ae3
Nov 03 16:22:22.850268 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:22.850234    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm" event={"ID":"24c2b816-5bf9-40f9-98d4-e325d16a869c","Type":"ContainerStarted","Data":"98e68074a3e7543a8b8de3904782001b5da49106deff7183548af1dced917ae3"}
Nov 03 16:22:23.855958 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:23.855924    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm" event={"ID":"24c2b816-5bf9-40f9-98d4-e325d16a869c","Type":"ContainerStarted","Data":"c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0"}
Nov 03 16:22:23.856420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:23.856056    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:27.764906 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:27.764865    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:22:27.766547 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:27.764933    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:22:27.879581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:27.879549    5605 generic.go:356] "Generic (PLEG): container finished" podID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerID="5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d" exitCode=0
Nov 03 16:22:27.879772 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:27.879603    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" event={"ID":"806a66bb-48eb-41ae-8fbe-c7869b6040f5","Type":"ContainerDied","Data":"5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d"}
Nov 03 16:22:27.880015 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:27.879999    5605 scope.go:122] "RemoveContainer" containerID="5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d"
Nov 03 16:22:27.881257 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:27.881220    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:27.881369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:27.881309    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:27.881369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:27.881358    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:27.979860 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:27.979626    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": read tcp 10.133.0.2:55504->10.133.0.34:15021: read: connection reset by peer" start-of-body=
Nov 03 16:22:27.979860 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:27.979792    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": read tcp 10.133.0.2:55504->10.133.0.34:15021: read: connection reset by peer"
Nov 03 16:22:28.085797 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:28.085717    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm" podStartSLOduration=7.053904594 podStartE2EDuration="8.085704554s" totalImagesPullingTime="1.03179996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:22:23.970549125 +0000 UTC m=+629.795000415" watchObservedRunningTime="2026-11-03 16:22:28.085704554 +0000 UTC m=+633.910155828"
Nov 03 16:22:28.911614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:28.911580    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" event={"ID":"806a66bb-48eb-41ae-8fbe-c7869b6040f5","Type":"ContainerStarted","Data":"009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377"}
Nov 03 16:22:28.913099 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:28.913075    5605 generic.go:356] "Generic (PLEG): container finished" podID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerID="6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1" exitCode=0
Nov 03 16:22:28.913232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:28.913146    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" event={"ID":"ae2b9d64-981e-419f-84be-3beae332c9d1","Type":"ContainerDied","Data":"6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1"}
Nov 03 16:22:28.913491 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:28.913468    5605 scope.go:122] "RemoveContainer" containerID="6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1"
Nov 03 16:22:28.914663 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:28.914636    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:28.914758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:28.914702    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:28.914806 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:28.914759    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:29.206816 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:29.195398    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:22:29.206816 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:29.196300    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 16:22:29.206816 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:29.196449    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 16:22:29.895753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:29.879956    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:22:29.951755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:29.944818    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" event={"ID":"ae2b9d64-981e-419f-84be-3beae332c9d1","Type":"ContainerStarted","Data":"05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c"}
Nov 03 16:22:30.280836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:30.265843    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:22:30.280836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:30.267194    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 16:22:30.280836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:30.267251    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 16:22:30.300754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:30.298152    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 03 16:22:30.300754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:30.298249    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 03 16:22:31.185802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.185766    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 03 16:22:31.186422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.186379    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 03 16:22:31.196107 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.196075    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 16:22:31.196322 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.196295    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 16:22:31.513173 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.513084    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"]
Nov 03 16:22:31.529754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.526041    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.572791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.572762    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"]
Nov 03 16:22:31.617754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.614293    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ae4568-03c9-444c-82e8-62bbbd007a94-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.617754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.614390    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj6ln\" (UniqueName: \"kubernetes.io/projected/63ae4568-03c9-444c-82e8-62bbbd007a94-kube-api-access-fj6ln\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.617754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.614464    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-metrics-certs\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.617754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.614518    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-maas-api-tls\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.652391 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.652345    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-42da0f-serving-cert\""
Nov 03 16:22:31.737936 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.737887    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-metrics-certs\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.738857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.738831    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-maas-api-tls\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.739523 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:22:31.739115    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-serving-cert: secret "maas-api-e2e-worker-w0-42da0f-serving-cert" not found
Nov 03 16:22:31.739523 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:22:31.739274    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-maas-api-tls podName:63ae4568-03c9-444c-82e8-62bbbd007a94 nodeName:}" failed. No retries permitted until 2026-11-03 16:22:32.239249887 +0000 UTC m=+638.063701159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-maas-api-tls") pod "maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42" (UID: "63ae4568-03c9-444c-82e8-62bbbd007a94") : secret "maas-api-e2e-worker-w0-42da0f-serving-cert" not found
Nov 03 16:22:31.739855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.739837    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ae4568-03c9-444c-82e8-62bbbd007a94-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.763717 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.740634    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ae4568-03c9-444c-82e8-62bbbd007a94-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.763717 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.763392    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-metrics-certs\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.766297 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.765825    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj6ln\" (UniqueName: \"kubernetes.io/projected/63ae4568-03c9-444c-82e8-62bbbd007a94-kube-api-access-fj6ln\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:31.872764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:31.871791    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj6ln\" (UniqueName: \"kubernetes.io/projected/63ae4568-03c9-444c-82e8-62bbbd007a94-kube-api-access-fj6ln\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:32.192767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:32.189091    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 03 16:22:32.192767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:32.189155    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 03 16:22:32.196758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:32.194045    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 16:22:32.196758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:32.194101    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 16:22:32.275909 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:32.275880    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-maas-api-tls\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:32.280419 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:32.278396    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-maas-api-tls\") pod \"maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:32.460965 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:32.460865    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:33.001454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:32.991552    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"]
Nov 03 16:22:33.189327 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:33.189288    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 03 16:22:33.189634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:33.189592    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 03 16:22:33.197720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:33.197675    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 16:22:33.198407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:33.198370    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 16:22:33.466691 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:22:33.466658    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:22:33.973447 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:33.973412    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42" event={"ID":"63ae4568-03c9-444c-82e8-62bbbd007a94","Type":"ContainerStarted","Data":"a334b0721c86d158121fdd9d134d3f8e32085ed06a9485175e7576b95d3b0b73"}
Nov 03 16:22:34.189162 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:34.189129    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 03 16:22:34.189403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:34.189375    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 03 16:22:34.197135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:34.197102    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 16:22:34.197351 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:34.197324    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 16:22:34.980091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:34.980052    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42" event={"ID":"63ae4568-03c9-444c-82e8-62bbbd007a94","Type":"ContainerStarted","Data":"3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9"}
Nov 03 16:22:34.980490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:34.980152    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:35.195238 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:35.195139    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Nov 03 16:22:35.195493 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:35.195465    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Nov 03 16:22:35.214821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:35.212858    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:22:35.214821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:35.214234    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:22:35.216048 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:35.215998    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:22:35.330094 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:35.330033    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42" podStartSLOduration=3.301795079 podStartE2EDuration="4.330012994s" totalImagesPullingTime="1.028217915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:22:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:22:35.094856045 +0000 UTC m=+640.919307319" watchObservedRunningTime="2026-11-03 16:22:35.330012994 +0000 UTC m=+641.154464267"
Nov 03 16:22:36.189202 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:36.189170    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:22:36.189662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:36.189443    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:22:36.190271 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:36.190254    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:22:38.002584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:38.002550    5605 generic.go:356] "Generic (PLEG): container finished" podID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerID="975a84732baa998eb1d16061be20b77a36f077f30bf0ef318db3e284981057af" exitCode=0
Nov 03 16:22:38.003040 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:38.002624    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" event={"ID":"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066","Type":"ContainerDied","Data":"975a84732baa998eb1d16061be20b77a36f077f30bf0ef318db3e284981057af"}
Nov 03 16:22:38.003183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:38.003168    5605 scope.go:122] "RemoveContainer" containerID="975a84732baa998eb1d16061be20b77a36f077f30bf0ef318db3e284981057af"
Nov 03 16:22:38.004477 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:38.004449    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:38.004536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:38.004516    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:38.004571 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:38.004547    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:39.011793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:39.011759    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" event={"ID":"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066","Type":"ContainerStarted","Data":"85807ecc93933e92dbdffdbac1b60541f94b80bd121599bdfabb11c715dd4615"}
Nov 03 16:22:39.525405 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:39.525345    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:22:39.525906 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:39.525862    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Nov 03 16:22:39.526112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:39.526089    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Nov 03 16:22:40.526243 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:40.526204    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Nov 03 16:22:40.527627 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:40.527595    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Nov 03 16:22:41.034987 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:41.034952    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 16:22:41.524200 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:41.524162    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Nov 03 16:22:41.524491 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:41.524464    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Nov 03 16:22:42.529752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:42.526143    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Nov 03 16:22:42.529752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:42.526207    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Nov 03 16:22:42.730878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:42.730668    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:22:43.525653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:43.525152    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Nov 03 16:22:43.525653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:43.525233    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Nov 03 16:22:44.527980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:44.527947    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:22:44.529038 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:44.528995    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:22:49.095429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:49.095391    5605 generic.go:356] "Generic (PLEG): container finished" podID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerID="e2c9c2e15e45c9815a9b258c622cba8c360020710b63b1a319dec8d5e4498a93" exitCode=0
Nov 03 16:22:49.095891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:49.095465    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" event={"ID":"bfd53f66-eba2-4ad7-a908-a42c9b6914d9","Type":"ContainerDied","Data":"e2c9c2e15e45c9815a9b258c622cba8c360020710b63b1a319dec8d5e4498a93"}
Nov 03 16:22:49.095891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:49.095848    5605 scope.go:122] "RemoveContainer" containerID="e2c9c2e15e45c9815a9b258c622cba8c360020710b63b1a319dec8d5e4498a93"
Nov 03 16:22:49.097065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:49.097035    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:49.097125 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:49.097106    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:49.097168 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:49.097137    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:22:50.111992 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:50.111953    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" event={"ID":"bfd53f66-eba2-4ad7-a908-a42c9b6914d9","Type":"ContainerStarted","Data":"b9fa72bfae38a8b4e3b1aafa3c5a5f460b3284d6cbd3752c2565b9b21f1e814b"}
Nov 03 16:22:50.869827 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:50.869792    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:22:50.870086 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:50.870064    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 03 16:22:50.870136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:50.870114    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 03 16:22:51.869939 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:51.869900    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 03 16:22:51.870427 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:51.869979    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 03 16:22:52.870018 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:52.869978    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 03 16:22:52.870391 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:52.870046    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 03 16:22:53.870161 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:53.870120    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 03 16:22:53.870567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:53.870180    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 03 16:22:54.870175 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:54.870140    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 03 16:22:54.870559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:54.870195    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 03 16:22:55.874111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:55.874082    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:22:55.874575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:55.874390    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:22:55.875136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:22:55.875119    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:23:03.391377 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:23:03.391302    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:23:06.051010 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.050966    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"]
Nov 03 16:23:06.053671 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.053643    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.150920 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.150891    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c88d54\"/\"default-dockercfg-wr497\""
Nov 03 16:23:06.151096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.150951    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c88d54\"/\"simulator-w2-c88d54-kserve-self-signed-certs\""
Nov 03 16:23:06.151096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.150973    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c88d54\"/\"openshift-service-ca.crt\""
Nov 03 16:23:06.151096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.151069    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c88d54\"/\"kube-root-ca.crt\""
Nov 03 16:23:06.152965 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.152948    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"]
Nov 03 16:23:06.175364 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.175328    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-home\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.175559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.175383    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-model-cache\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.175559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.175500    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-tmp-dir\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.175665 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.175566    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvrbp\" (UniqueName: \"kubernetes.io/projected/e65c45be-2a14-4d0f-b618-69d01cb6926c-kube-api-access-bvrbp\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.175665 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.175587    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e65c45be-2a14-4d0f-b618-69d01cb6926c-tls-certs\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.175765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.175675    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-dshm\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.276957 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.276923    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-model-cache\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.277129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.276964    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-tmp-dir\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.277129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.277011    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvrbp\" (UniqueName: \"kubernetes.io/projected/e65c45be-2a14-4d0f-b618-69d01cb6926c-kube-api-access-bvrbp\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.277129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.277030    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e65c45be-2a14-4d0f-b618-69d01cb6926c-tls-certs\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.277129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.277065    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-dshm\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.277293 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.277143    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-home\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.277408 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.277387    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-model-cache\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.277446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.277400    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-tmp-dir\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.277493 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.277443    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-home\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.279575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.279552    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-dshm\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.279898 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.279875    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e65c45be-2a14-4d0f-b618-69d01cb6926c-tls-certs\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.340563 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.340532    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvrbp\" (UniqueName: \"kubernetes.io/projected/e65c45be-2a14-4d0f-b618-69d01cb6926c-kube-api-access-bvrbp\") pod \"simulator-w2-c88d54-kserve-75899c4c6c-whx6f\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") " pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.364463 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.364428    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:06.742310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:06.742233    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"]
Nov 03 16:23:06.748874 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:23:06.748837    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode65c45be_2a14_4d0f_b618_69d01cb6926c.slice/crio-55779d38190192a263da552f79bb152257a364ec10225bdd2230f60655ce92da WatchSource:0}: Error finding container 55779d38190192a263da552f79bb152257a364ec10225bdd2230f60655ce92da: Status 404 returned error can't find the container with id 55779d38190192a263da552f79bb152257a364ec10225bdd2230f60655ce92da
Nov 03 16:23:07.244115 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:07.244081    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" event={"ID":"e65c45be-2a14-4d0f-b618-69d01cb6926c","Type":"ContainerStarted","Data":"85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180"}
Nov 03 16:23:07.244115 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:07.244121    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" event={"ID":"e65c45be-2a14-4d0f-b618-69d01cb6926c","Type":"ContainerStarted","Data":"55779d38190192a263da552f79bb152257a364ec10225bdd2230f60655ce92da"}
Nov 03 16:23:07.359340 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:07.359277    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podStartSLOduration=1.359257837 podStartE2EDuration="1.359257837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:23:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:23:07.35748003 +0000 UTC m=+673.181931302" watchObservedRunningTime="2026-11-03 16:23:07.359257837 +0000 UTC m=+673.183709110"
Nov 03 16:23:07.365609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:07.365579    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:16.365618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:16.365573    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:16.378844 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:16.378818    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:17.306671 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:17.306644    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:23:33.391053 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:23:33.391013    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:23:44.775878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:44.775836    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:23:44.776303 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:44.775908    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:23:46.453689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:46.453653    5605 generic.go:356] "Generic (PLEG): container finished" podID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerID="8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5" exitCode=0
Nov 03 16:23:46.454143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:46.453744    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" event={"ID":"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a","Type":"ContainerDied","Data":"8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5"}
Nov 03 16:23:46.454188 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:46.454172    5605 scope.go:122] "RemoveContainer" containerID="8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5"
Nov 03 16:23:46.455509 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:46.455482    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:23:46.455581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:46.455554    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:23:46.455625 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:46.455587    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:23:47.462281 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:47.462246    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" event={"ID":"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a","Type":"ContainerStarted","Data":"223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4"}
Nov 03 16:23:47.510515 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:47.510468    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:23:47.510711 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:47.510696    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 03 16:23:47.510797 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:47.510760    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 03 16:23:48.510403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:48.510370    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 03 16:23:48.510821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:48.510423    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 03 16:23:49.510407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:49.510370    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 03 16:23:49.510589 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:49.510425    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 03 16:23:50.510845 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:50.510798    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 03 16:23:50.511379 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:50.510865    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 03 16:23:51.510361 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:51.510327    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 03 16:23:51.510547 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:51.510380    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 03 16:23:52.510055 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:52.510023    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Nov 03 16:23:52.510459 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:52.510075    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Nov 03 16:23:53.513872 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:53.513841    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:23:53.514272 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:53.514127    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:23:53.514822 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:53.514805    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:23:54.447059 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.447023    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"]
Nov 03 16:23:54.449903 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.449876    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.482486 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.482453    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tmp-dir\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.482651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.482529    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-home\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.482651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.482632    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-model-cache\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.482793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.482748    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tls-certs\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.482793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.482771    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-dshm\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.482865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.482794    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4xg9\" (UniqueName: \"kubernetes.io/projected/464d2b0b-6212-43b3-b182-7f860bfe2e4f-kube-api-access-q4xg9\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.546131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.546102    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2bb620\"/\"default-dockercfg-rwp4d\""
Nov 03 16:23:54.546131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.546102    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2bb620\"/\"simulator-w4-2bb620-kserve-self-signed-certs\""
Nov 03 16:23:54.546131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.546107    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-2bb620\"/\"openshift-service-ca.crt\""
Nov 03 16:23:54.546574 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.546161    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-2bb620\"/\"kube-root-ca.crt\""
Nov 03 16:23:54.548659 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.548641    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"]
Nov 03 16:23:54.583461 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.583436    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tls-certs\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.583461 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.583466    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-dshm\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.583777 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.583489    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4xg9\" (UniqueName: \"kubernetes.io/projected/464d2b0b-6212-43b3-b182-7f860bfe2e4f-kube-api-access-q4xg9\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.583777 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.583536    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tmp-dir\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.583777 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.583619    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-home\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.583777 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.583654    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-model-cache\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.584023 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.583997    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tmp-dir\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.584098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.584076    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-home\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.584164 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.584147    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-model-cache\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.586682 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.586659    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tls-certs\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.586984 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.586959    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-dshm\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.639880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.639844    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4xg9\" (UniqueName: \"kubernetes.io/projected/464d2b0b-6212-43b3-b182-7f860bfe2e4f-kube-api-access-q4xg9\") pod \"simulator-w4-2bb620-kserve-75c9994b57-zclvw\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") " pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:54.763674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:54.763578    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:23:55.140125 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:55.140090    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"]
Nov 03 16:23:55.146621 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:23:55.146591    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod464d2b0b_6212_43b3_b182_7f860bfe2e4f.slice/crio-5fb50fe8b5e971272143835bf0d3e117bd6f5d49a45d74e0fdf9f22f7e76f07b WatchSource:0}: Error finding container 5fb50fe8b5e971272143835bf0d3e117bd6f5d49a45d74e0fdf9f22f7e76f07b: Status 404 returned error can't find the container with id 5fb50fe8b5e971272143835bf0d3e117bd6f5d49a45d74e0fdf9f22f7e76f07b
Nov 03 16:23:55.506784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:55.506641    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" event={"ID":"464d2b0b-6212-43b3-b182-7f860bfe2e4f","Type":"ContainerStarted","Data":"b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54"}
Nov 03 16:23:55.506784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:55.506690    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" event={"ID":"464d2b0b-6212-43b3-b182-7f860bfe2e4f","Type":"ContainerStarted","Data":"5fb50fe8b5e971272143835bf0d3e117bd6f5d49a45d74e0fdf9f22f7e76f07b"}
Nov 03 16:23:55.619138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:55.619088    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podStartSLOduration=1.619070472 podStartE2EDuration="1.619070472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:23:55.617771175 +0000 UTC m=+721.442222443" watchObservedRunningTime="2026-11-03 16:23:55.619070472 +0000 UTC m=+721.443521745"
Nov 03 16:23:55.763838 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:23:55.763761    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:24:03.391155 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:24:03.391120    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:24:04.764697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:04.764658    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:24:04.777098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:04.777073    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:24:05.568131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:05.568097    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:24:06.308846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.308816    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"]
Nov 03 16:24:06.311413 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.311394    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.367293 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.367260    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"]
Nov 03 16:24:06.408851 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.408815    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-tmp-dir\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.408851 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.408855    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-model-cache\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.409084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.408880    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92pcz\" (UniqueName: \"kubernetes.io/projected/33b210d8-b8be-4b1a-b366-166dcec474ca-kube-api-access-92pcz\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.409084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.409025    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-home\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.409084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.409066    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b210d8-b8be-4b1a-b366-166dcec474ca-tls-certs\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.409251 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.409228    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-dshm\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.414295 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.414273    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-6ae690\"/\"kube-root-ca.crt\""
Nov 03 16:24:06.415878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.415860    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-6ae690\"/\"openshift-service-ca.crt\""
Nov 03 16:24:06.416372 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.416357    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-6ae690\"/\"premium-w5-6ae690-kserve-self-signed-certs\""
Nov 03 16:24:06.417397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.417383    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-6ae690\"/\"default-dockercfg-w9645\""
Nov 03 16:24:06.510362 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.510329    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-home\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.510562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.510372    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b210d8-b8be-4b1a-b366-166dcec474ca-tls-certs\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.510562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.510468    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-dshm\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.510562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.510513    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-tmp-dir\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.510562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.510531    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-model-cache\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.510562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.510560    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92pcz\" (UniqueName: \"kubernetes.io/projected/33b210d8-b8be-4b1a-b366-166dcec474ca-kube-api-access-92pcz\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.510861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.510799    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-home\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.510919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.510901    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-tmp-dir\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.510984 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.510961    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-model-cache\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.512982 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.512957    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-dshm\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.513265 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.513243    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b210d8-b8be-4b1a-b366-166dcec474ca-tls-certs\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.572770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.572686    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92pcz\" (UniqueName: \"kubernetes.io/projected/33b210d8-b8be-4b1a-b366-166dcec474ca-kube-api-access-92pcz\") pod \"premium-w5-6ae690-kserve-759944fcb-zhjvc\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") " pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:06.623555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:06.623522    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:07.036623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:07.036595    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"]
Nov 03 16:24:07.038683 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:24:07.038651    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33b210d8_b8be_4b1a_b366_166dcec474ca.slice/crio-d1a8f7edfe18f71448ff78bfdd7ba76eb7f489941137084c48d10798bf723f57 WatchSource:0}: Error finding container d1a8f7edfe18f71448ff78bfdd7ba76eb7f489941137084c48d10798bf723f57: Status 404 returned error can't find the container with id d1a8f7edfe18f71448ff78bfdd7ba76eb7f489941137084c48d10798bf723f57
Nov 03 16:24:07.569907 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:07.569864    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" event={"ID":"33b210d8-b8be-4b1a-b366-166dcec474ca","Type":"ContainerStarted","Data":"04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b"}
Nov 03 16:24:07.569907 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:07.569914    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" event={"ID":"33b210d8-b8be-4b1a-b366-166dcec474ca","Type":"ContainerStarted","Data":"d1a8f7edfe18f71448ff78bfdd7ba76eb7f489941137084c48d10798bf723f57"}
Nov 03 16:24:07.624062 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:07.624026    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:07.693072 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:07.693015    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podStartSLOduration=1.6929998450000001 podStartE2EDuration="1.692999845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:24:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:24:07.692439475 +0000 UTC m=+733.516890748" watchObservedRunningTime="2026-11-03 16:24:07.692999845 +0000 UTC m=+733.517451139"
Nov 03 16:24:16.624125 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:16.624089    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:16.636793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:16.636768    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:17.631066 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:17.631042    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:24:33.392526 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:24:33.392442    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:24:51.866283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.866245    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"]
Nov 03 16:24:51.869413 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.869391    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:51.902563 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.902535    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-tmp-dir\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:51.902713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.902569    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-home\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:51.902713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.902674    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-model-cache\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:51.902837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.902755    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25xpq\" (UniqueName: \"kubernetes.io/projected/7e014c82-88ba-42be-9a82-ae0367902465-kube-api-access-25xpq\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:51.902892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.902869    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-dshm\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:51.902929 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.902914    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e014c82-88ba-42be-9a82-ae0367902465-tls-certs\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:51.976474 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.976442    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-192564\"/\"openshift-service-ca.crt\""
Nov 03 16:24:51.976636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.976523    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-192564\"/\"kube-root-ca.crt\""
Nov 03 16:24:51.976636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.976559    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-192564\"/\"default-dockercfg-vxsb2\""
Nov 03 16:24:51.976937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.976916    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-192564\"/\"premium-w3-192564-kserve-self-signed-certs\""
Nov 03 16:24:51.982184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:51.982167    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"]
Nov 03 16:24:52.004900 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.004863    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25xpq\" (UniqueName: \"kubernetes.io/projected/7e014c82-88ba-42be-9a82-ae0367902465-kube-api-access-25xpq\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.005523 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.005497    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-dshm\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.006065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.006045    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e014c82-88ba-42be-9a82-ae0367902465-tls-certs\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.007338 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.007182    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-tmp-dir\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.008194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.008175    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-home\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.020065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.020035    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-model-cache\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.026159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.025977    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-tmp-dir\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.026361 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.025989    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-home\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.026467 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.026199    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-model-cache\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.031922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.031567    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-dshm\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.031922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.031635    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e014c82-88ba-42be-9a82-ae0367902465-tls-certs\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.067540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.067506    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25xpq\" (UniqueName: \"kubernetes.io/projected/7e014c82-88ba-42be-9a82-ae0367902465-kube-api-access-25xpq\") pod \"premium-w3-192564-kserve-58cff55cbd-x75wj\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") " pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.180629 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.180550    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:24:52.561467 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.561444    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"]
Nov 03 16:24:52.562927 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:24:52.562885    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e014c82_88ba_42be_9a82_ae0367902465.slice/crio-cdb496db3951a1ad806aab3cd5ba0f8bb9262fdf89a7d7cb4f02313f1a383cb2 WatchSource:0}: Error finding container cdb496db3951a1ad806aab3cd5ba0f8bb9262fdf89a7d7cb4f02313f1a383cb2: Status 404 returned error can't find the container with id cdb496db3951a1ad806aab3cd5ba0f8bb9262fdf89a7d7cb4f02313f1a383cb2
Nov 03 16:24:52.784773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.784720    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" event={"ID":"7e014c82-88ba-42be-9a82-ae0367902465","Type":"ContainerStarted","Data":"5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6"}
Nov 03 16:24:52.784942 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.784787    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" event={"ID":"7e014c82-88ba-42be-9a82-ae0367902465","Type":"ContainerStarted","Data":"cdb496db3951a1ad806aab3cd5ba0f8bb9262fdf89a7d7cb4f02313f1a383cb2"}
Nov 03 16:24:52.909736 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:52.909666    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podStartSLOduration=1.909651132 podStartE2EDuration="1.909651132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:24:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:24:52.908573694 +0000 UTC m=+778.733024962" watchObservedRunningTime="2026-11-03 16:24:52.909651132 +0000 UTC m=+778.734102402"
Nov 03 16:24:53.181028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:24:53.180945    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:25:02.180788 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:02.180752    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:25:02.193484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:02.193447    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:25:02.845008 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:02.844975    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:25:03.392107 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:25:03.392068    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:25:28.100017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.099977    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"]
Nov 03 16:25:28.102665 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.102648    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.214447 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.214405    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-2bb620\"/\"premium-w4-2bb620-kserve-self-signed-certs\""
Nov 03 16:25:28.216231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.216207    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"]
Nov 03 16:25:28.233032 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.232990    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-dshm\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.233205 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.233067    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-home\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.233205 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.233118    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg999\" (UniqueName: \"kubernetes.io/projected/075e5aa1-a616-42dc-8c95-62f8db2c1194-kube-api-access-fg999\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.233205 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.233175    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/075e5aa1-a616-42dc-8c95-62f8db2c1194-tls-certs\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.233380 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.233208    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-tmp-dir\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.233380 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.233251    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-model-cache\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.334148 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.334115    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-tmp-dir\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.334362 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.334170    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-model-cache\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.334362 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.334216    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-dshm\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.334362 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.334255    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-home\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.334362 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.334294    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg999\" (UniqueName: \"kubernetes.io/projected/075e5aa1-a616-42dc-8c95-62f8db2c1194-kube-api-access-fg999\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.334362 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.334331    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/075e5aa1-a616-42dc-8c95-62f8db2c1194-tls-certs\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.334702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.334679    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-model-cache\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.334832 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.334712    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-home\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.334832 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.334683    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-tmp-dir\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.336673 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.336652    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-dshm\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.336926 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.336910    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/075e5aa1-a616-42dc-8c95-62f8db2c1194-tls-certs\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.395911 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.395840    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg999\" (UniqueName: \"kubernetes.io/projected/075e5aa1-a616-42dc-8c95-62f8db2c1194-kube-api-access-fg999\") pod \"premium-w4-2bb620-kserve-74dd78bc88-s6tl6\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") " pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.413849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.413820    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:28.793142 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.793112    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"]
Nov 03 16:25:28.795243 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:25:28.795214    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod075e5aa1_a616_42dc_8c95_62f8db2c1194.slice/crio-f61f88dec809422971defe4d91b601b622da8992f24aa0ee516776eb508cddcd WatchSource:0}: Error finding container f61f88dec809422971defe4d91b601b622da8992f24aa0ee516776eb508cddcd: Status 404 returned error can't find the container with id f61f88dec809422971defe4d91b601b622da8992f24aa0ee516776eb508cddcd
Nov 03 16:25:28.967120 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.967084    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" event={"ID":"075e5aa1-a616-42dc-8c95-62f8db2c1194","Type":"ContainerStarted","Data":"030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8"}
Nov 03 16:25:28.967120 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:28.967128    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" event={"ID":"075e5aa1-a616-42dc-8c95-62f8db2c1194","Type":"ContainerStarted","Data":"f61f88dec809422971defe4d91b601b622da8992f24aa0ee516776eb508cddcd"}
Nov 03 16:25:29.091490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:29.091434    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podStartSLOduration=1.091419047 podStartE2EDuration="1.091419047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:25:29.090296654 +0000 UTC m=+814.914747927" watchObservedRunningTime="2026-11-03 16:25:29.091419047 +0000 UTC m=+814.915870326"
Nov 03 16:25:29.414789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:29.414672    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:33.391018 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:25:33.390969    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:25:38.414643 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:38.414601    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:38.427408 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:38.427376    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:25:39.026674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:25:39.026646    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:26:03.391437 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:26:03.391352    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:26:18.751524 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:18.751488    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"]
Nov 03 16:26:18.752053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:18.751762    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerName="istio-proxy" containerID="cri-o://8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0" gracePeriod=30
Nov 03 16:26:24.006982 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.006954    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:26:24.144132 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144102    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-envoy\") pod \"03f9ac99-4327-4059-942b-f1ce28a36b72\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") "
Nov 03 16:26:24.144305 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144151    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-certs\") pod \"03f9ac99-4327-4059-942b-f1ce28a36b72\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") "
Nov 03 16:26:24.144305 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144178    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03f9ac99-4327-4059-942b-f1ce28a36b72-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03f9ac99-4327-4059-942b-f1ce28a36b72-istiod-ca-cert\") pod \"03f9ac99-4327-4059-942b-f1ce28a36b72\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") "
Nov 03 16:26:24.144305 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144206    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-podinfo\") pod \"03f9ac99-4327-4059-942b-f1ce28a36b72\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") "
Nov 03 16:26:24.144305 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144232    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-socket\") pod \"03f9ac99-4327-4059-942b-f1ce28a36b72\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") "
Nov 03 16:26:24.144305 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144255    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-kube-api-access-4djxb\" (UniqueName: \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-kube-api-access-4djxb\") pod \"03f9ac99-4327-4059-942b-f1ce28a36b72\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") "
Nov 03 16:26:24.144584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144346    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-credential-socket\") pod \"03f9ac99-4327-4059-942b-f1ce28a36b72\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") "
Nov 03 16:26:24.144584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144407    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-data\") pod \"03f9ac99-4327-4059-942b-f1ce28a36b72\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") "
Nov 03 16:26:24.144584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144433    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-token\" (UniqueName: \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-token\") pod \"03f9ac99-4327-4059-942b-f1ce28a36b72\" (UID: \"03f9ac99-4327-4059-942b-f1ce28a36b72\") "
Nov 03 16:26:24.144584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144466    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-certs" pod "03f9ac99-4327-4059-942b-f1ce28a36b72" (UID: "03f9ac99-4327-4059-942b-f1ce28a36b72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:26:24.144804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144576    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-socket" pod "03f9ac99-4327-4059-942b-f1ce28a36b72" (UID: "03f9ac99-4327-4059-942b-f1ce28a36b72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:26:24.144804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144779    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-credential-socket" pod "03f9ac99-4327-4059-942b-f1ce28a36b72" (UID: "03f9ac99-4327-4059-942b-f1ce28a36b72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:26:24.144896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144855    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03f9ac99-4327-4059-942b-f1ce28a36b72-istiod-ca-cert" pod "03f9ac99-4327-4059-942b-f1ce28a36b72" (UID: "03f9ac99-4327-4059-942b-f1ce28a36b72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:26:24.144896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.144857    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-data" pod "03f9ac99-4327-4059-942b-f1ce28a36b72" (UID: "03f9ac99-4327-4059-942b-f1ce28a36b72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:26:24.145066 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.145044    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:26:24.145066 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.145067    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:26:24.145066 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.145077    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:26:24.145220 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.145089    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03f9ac99-4327-4059-942b-f1ce28a36b72-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:26:24.145220 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.145101    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:26:24.146618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.146586    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-podinfo" pod "03f9ac99-4327-4059-942b-f1ce28a36b72" (UID: "03f9ac99-4327-4059-942b-f1ce28a36b72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:26:24.146894 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.146877    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-kube-api-access-4djxb" pod "03f9ac99-4327-4059-942b-f1ce28a36b72" (UID: "03f9ac99-4327-4059-942b-f1ce28a36b72"). InnerVolumeSpecName "kube-api-access-4djxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:26:24.147160 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.147134    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-envoy" pod "03f9ac99-4327-4059-942b-f1ce28a36b72" (UID: "03f9ac99-4327-4059-942b-f1ce28a36b72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:26:24.147160 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.147153    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-token" pod "03f9ac99-4327-4059-942b-f1ce28a36b72" (UID: "03f9ac99-4327-4059-942b-f1ce28a36b72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:26:24.246294 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.246249    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:26:24.246294 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.246286    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4djxb\" (UniqueName: \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-kube-api-access-4djxb\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:26:24.246294 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.246301    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:26:24.246537 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.246313    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03f9ac99-4327-4059-942b-f1ce28a36b72-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:26:24.266488 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.266450    5605 generic.go:356] "Generic (PLEG): container finished" podID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerID="8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0" exitCode=0
Nov 03 16:26:24.266673 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.266511    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"
Nov 03 16:26:24.266673 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.266548    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" event={"ID":"03f9ac99-4327-4059-942b-f1ce28a36b72","Type":"ContainerDied","Data":"8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0"}
Nov 03 16:26:24.266673 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.266658    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8" event={"ID":"03f9ac99-4327-4059-942b-f1ce28a36b72","Type":"ContainerDied","Data":"6262d5500f0d834e31aa58128dbb2b378b7d015ee285667d49c30242b5cdd023"}
Nov 03 16:26:24.266900 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.266696    5605 scope.go:122] "RemoveContainer" containerID="8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0"
Nov 03 16:26:24.277200 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.277172    5605 scope.go:122] "RemoveContainer" containerID="8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0"
Nov 03 16:26:24.277578 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:26:24.277508    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0\": container with ID starting with 8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0 not found: ID does not exist" containerID="8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0"
Nov 03 16:26:24.277578 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.277547    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0"} err="failed to get container status \"8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0\": rpc error: code = NotFound desc = could not find container \"8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0\": container with ID starting with 8c993ca985854a1459da21175b444624fab3a2d5fa07f68bd4df821a1c75eaa0 not found: ID does not exist"
Nov 03 16:26:24.444211 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.444129    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"]
Nov 03 16:26:24.453763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:24.453714    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-sl8t8"]
Nov 03 16:26:26.307166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:26.307137    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" path="/var/lib/kubelet/pods/03f9ac99-4327-4059-942b-f1ce28a36b72/volumes"
Nov 03 16:26:33.390637 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:26:33.390598    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:26:54.400777 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:54.400719    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:26:54.402955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:26:54.402930    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:27:03.391644 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:27:03.391601    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:27:17.334253 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.334216    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"]
Nov 03 16:27:17.334821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.334802    5605 cpu_manager.go:414] "containerMap: removing container" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerName="istio-proxy"
Nov 03 16:27:17.334912 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.334822    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerName="istio-proxy"
Nov 03 16:27:17.334993 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.334980    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="03f9ac99-4327-4059-942b-f1ce28a36b72" containerName="istio-proxy"
Nov 03 16:27:17.337678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.337655    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.396383 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.396358    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-547a38\"/\"default-dockercfg-2nfhm\""
Nov 03 16:27:17.405984 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.405955    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d0545c6-3e85-47d4-bf3e-b229191e424a-tls-certs\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.406129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.406100    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kkwv\" (UniqueName: \"kubernetes.io/projected/7d0545c6-3e85-47d4-bf3e-b229191e424a-kube-api-access-7kkwv\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.406196 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.406167    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-dshm\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.406287 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.406265    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-model-cache\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.406329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.406301    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-tmp-dir\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.406329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.406322    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-home\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.447874 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.447825    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-547a38\"/\"distinct-w6-547a38-kserve-self-signed-certs\""
Nov 03 16:27:17.449658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.449627    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"]
Nov 03 16:27:17.458163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.458132    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-547a38\"/\"openshift-service-ca.crt\""
Nov 03 16:27:17.458552 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.458530    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-547a38\"/\"kube-root-ca.crt\""
Nov 03 16:27:17.507582 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.507547    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kkwv\" (UniqueName: \"kubernetes.io/projected/7d0545c6-3e85-47d4-bf3e-b229191e424a-kube-api-access-7kkwv\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.507821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.507597    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-dshm\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.507821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.507630    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-model-cache\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.507821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.507646    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-tmp-dir\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.507821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.507664    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-home\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.507821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.507717    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d0545c6-3e85-47d4-bf3e-b229191e424a-tls-certs\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.508163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.508142    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-model-cache\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.508163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.508155    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-tmp-dir\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.508275 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.508205    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-home\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.510358 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.510331    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-dshm\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.510600 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.510584    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d0545c6-3e85-47d4-bf3e-b229191e424a-tls-certs\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.569509 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.569477    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kkwv\" (UniqueName: \"kubernetes.io/projected/7d0545c6-3e85-47d4-bf3e-b229191e424a-kube-api-access-7kkwv\") pod \"distinct-w6-547a38-kserve-66765cf47c-p5v6v\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:17.649914 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:17.649877    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:18.052456 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:18.052429    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"]
Nov 03 16:27:18.054483 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:27:18.054454    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d0545c6_3e85_47d4_bf3e_b229191e424a.slice/crio-60e7bfa83e9773c4b7796aa7de33d855a417d01092482b1c13704b73fc4961e3 WatchSource:0}: Error finding container 60e7bfa83e9773c4b7796aa7de33d855a417d01092482b1c13704b73fc4961e3: Status 404 returned error can't find the container with id 60e7bfa83e9773c4b7796aa7de33d855a417d01092482b1c13704b73fc4961e3
Nov 03 16:27:18.539471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:18.539432    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" event={"ID":"7d0545c6-3e85-47d4-bf3e-b229191e424a","Type":"ContainerStarted","Data":"ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688"}
Nov 03 16:27:18.539471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:18.539475    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" event={"ID":"7d0545c6-3e85-47d4-bf3e-b229191e424a","Type":"ContainerStarted","Data":"60e7bfa83e9773c4b7796aa7de33d855a417d01092482b1c13704b73fc4961e3"}
Nov 03 16:27:18.650701 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:18.650663    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:18.661450 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:18.661404    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podStartSLOduration=1.661392462 podStartE2EDuration="1.661392462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:27:18.660930239 +0000 UTC m=+924.485381512" watchObservedRunningTime="2026-11-03 16:27:18.661392462 +0000 UTC m=+924.485843732"
Nov 03 16:27:19.261654 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.261619    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"]
Nov 03 16:27:19.265012 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.264987    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.326817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.326783    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.326981 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.326909    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.326981 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.326962    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.327126 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.326999    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.327126 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.327069    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.327126 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.327103    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.327288 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.327144    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.327288 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.327250    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.327396 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.327312    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj9xw\" (UniqueName: \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-kube-api-access-sj9xw\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.373369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.373330    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"]
Nov 03 16:27:19.428756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.428709    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.428932 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.428781    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.428932 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.428832    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.428932 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.428851    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.428932 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.428867    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.428932 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.428902    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.429187 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.428941    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj9xw\" (UniqueName: \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-kube-api-access-sj9xw\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.429187 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.429006    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.429187 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.429110    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.429187 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.429146    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.429187 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.429180    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.429422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.429214    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.429487 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.429468    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.429596 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.429578    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.431334 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.431316    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.431589 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.431574    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.492826 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.492797    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.493632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.493607    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj9xw\" (UniqueName: \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-kube-api-access-sj9xw\") pod \"maas-default-gateway-openshift-default-5bc884dd87-b2zrq\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:19.577878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:19.577795    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:20.012430 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:20.012403    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"]
Nov 03 16:27:20.014613 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:27:20.014582    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7e3f2a_1dff_4dad_8953_08b9337c6140.slice/crio-932027a629293d6e5562d2d7c896bae27a9f45862ea17be1b1313fcd232f7da7 WatchSource:0}: Error finding container 932027a629293d6e5562d2d7c896bae27a9f45862ea17be1b1313fcd232f7da7: Status 404 returned error can't find the container with id 932027a629293d6e5562d2d7c896bae27a9f45862ea17be1b1313fcd232f7da7
Nov 03 16:27:20.017152 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:20.017119    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:27:20.017250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:20.017184    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:27:20.017250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:20.017213    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:27:20.550157 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:20.550117    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" event={"ID":"0b7e3f2a-1dff-4dad-8953-08b9337c6140","Type":"ContainerStarted","Data":"267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0"}
Nov 03 16:27:20.550308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:20.550176    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" event={"ID":"0b7e3f2a-1dff-4dad-8953-08b9337c6140","Type":"ContainerStarted","Data":"932027a629293d6e5562d2d7c896bae27a9f45862ea17be1b1313fcd232f7da7"}
Nov 03 16:27:20.578886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:20.578861    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:20.730641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:20.730571    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" podStartSLOduration=1.730551756 podStartE2EDuration="1.730551756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:27:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:27:20.728270576 +0000 UTC m=+926.552721852" watchObservedRunningTime="2026-11-03 16:27:20.730551756 +0000 UTC m=+926.555003031"
Nov 03 16:27:21.580806 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:21.580766    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-b2zrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 03 16:27:21.581395 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:21.581363    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 03 16:27:22.578611 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:22.578579    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-b2zrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 03 16:27:22.578825 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:22.578630    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 03 16:27:23.578595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:23.578564    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-b2zrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 03 16:27:23.578997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:23.578614    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 03 16:27:24.578773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:24.578743    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-b2zrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 03 16:27:24.579165 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:24.578791    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 03 16:27:25.579027 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:25.578995    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-b2zrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 03 16:27:25.579672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:25.579046    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 03 16:27:26.579077 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:26.579043    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-b2zrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 03 16:27:26.579477 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:26.579119    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 03 16:27:27.578837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:27.578797    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-b2zrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Nov 03 16:27:27.579044 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:27.578867    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Nov 03 16:27:27.651040 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:27.651008    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:27.663347 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:27.663321    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:28.257239 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.257194    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"]
Nov 03 16:27:28.261584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.261558    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.319961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.319929    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"]
Nov 03 16:27:28.364381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.364344    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjnmx\" (UniqueName: \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-kube-api-access-vjnmx\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.364700 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.364677    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-data\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.382386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.368022    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-192564-openshift-default-dockercfg-256th\""
Nov 03 16:27:28.399763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.387837    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-certs\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.399763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.387884    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istiod-ca-cert\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.399763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.387921    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-token\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.399763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.387977    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-envoy\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.399763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.388033    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-podinfo\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.399763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.388079    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.399763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.388136    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-credential-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.489798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.489685    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjnmx\" (UniqueName: \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-kube-api-access-vjnmx\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.489975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.489868    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-data\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.489975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.489929    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-certs\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.489975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.489958    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istiod-ca-cert\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.490151 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.490008    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-token\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.490151 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.490063    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-envoy\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.490151 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.490137    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-podinfo\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.490301 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.490206    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.490301 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.490264    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-credential-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.490301 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.490275    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-data\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.490449 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.490323    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-certs\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.490641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.490616    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-credential-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.490918 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.490894    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.491591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.491568    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istiod-ca-cert\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.492919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.492899    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-podinfo\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.493031 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.493008    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-envoy\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.552808 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.552708    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-token\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.552936 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.552853    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjnmx\" (UniqueName: \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-kube-api-access-vjnmx\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.576136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.576104    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:28.616069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.616043    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:27:28.692795 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.692751    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:28.693871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.693823    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:28.695776 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:28.695184    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:27:29.037331 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:29.037309    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"]
Nov 03 16:27:29.038856 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:27:29.038819    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5666ee98_cdcd_4ce9_8eb1_cfa5f5d9c6c7.slice/crio-2aebb24a89fe49b80d0beeb7c17e0ac1501b54b02ba45ffc31988098ae291d2c WatchSource:0}: Error finding container 2aebb24a89fe49b80d0beeb7c17e0ac1501b54b02ba45ffc31988098ae291d2c: Status 404 returned error can't find the container with id 2aebb24a89fe49b80d0beeb7c17e0ac1501b54b02ba45ffc31988098ae291d2c
Nov 03 16:27:29.041237 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:29.041201    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:27:29.041332 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:29.041269    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:27:29.041332 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:29.041300    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:27:29.616851 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:29.616814    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" event={"ID":"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7","Type":"ContainerStarted","Data":"ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a"}
Nov 03 16:27:29.617148 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:29.617118    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" event={"ID":"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7","Type":"ContainerStarted","Data":"2aebb24a89fe49b80d0beeb7c17e0ac1501b54b02ba45ffc31988098ae291d2c"}
Nov 03 16:27:29.743964 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:29.743901    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" podStartSLOduration=1.743880025 podStartE2EDuration="1.743880025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:27:29.740679387 +0000 UTC m=+935.565130659" watchObservedRunningTime="2026-11-03 16:27:29.743880025 +0000 UTC m=+935.568331298"
Nov 03 16:27:30.577066 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:30.577028    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:30.578743 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:30.578695    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 03 16:27:30.578867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:30.578769    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" podUID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 03 16:27:31.576655 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:31.576625    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 03 16:27:31.577071 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:31.576679    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" podUID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 03 16:27:32.576680 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:32.576644    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 03 16:27:32.577127 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:32.576701    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" podUID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 03 16:27:33.413707 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:27:33.413669    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:27:33.577192 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:33.577158    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 03 16:27:33.577595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:33.577222    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" podUID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 03 16:27:34.420802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.420772    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"]
Nov 03 16:27:34.424585 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.424564    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.470670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.470631    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.470858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.470694    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.470858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.470804    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edac80ab-4b43-431a-b42c-c23c7232e43e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.470983 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.470866    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.470983 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.470920    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.470983 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.470958    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4nms\" (UniqueName: \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-kube-api-access-h4nms\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.471123 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.470988    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.471123 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.471026    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.471224 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.471156    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.513970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.513937    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"]
Nov 03 16:27:34.572116 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.572078    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.572316 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.572140    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.572316 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.572171    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edac80ab-4b43-431a-b42c-c23c7232e43e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.572316 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.572218    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.572316 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.572270    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.572316 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.572305    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4nms\" (UniqueName: \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-kube-api-access-h4nms\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.572618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.572337    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.572618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.572371    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.572618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.572410    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.572874 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.572841    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edac80ab-4b43-431a-b42c-c23c7232e43e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.573026 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.573004    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.573343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.573317    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.573432 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.573392    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.573518 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.573493    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.575120 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.575099    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.575571 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.575549    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.576672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.576652    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 03 16:27:34.576784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.576747    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" podUID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 03 16:27:34.634333 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.634307    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.634682 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.634499    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4nms\" (UniqueName: \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-kube-api-access-h4nms\") pod \"maas-default-gateway-openshift-default-5bc884dd87-59bv8\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:34.738053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:34.737957    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:35.198254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.198228    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"]
Nov 03 16:27:35.201255 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:27:35.201223    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedac80ab_4b43_431a_b42c_c23c7232e43e.slice/crio-2b0ece77041f37ebadae12c345d7c1c252f0e41ff78c0ea56676029c5034fec2 WatchSource:0}: Error finding container 2b0ece77041f37ebadae12c345d7c1c252f0e41ff78c0ea56676029c5034fec2: Status 404 returned error can't find the container with id 2b0ece77041f37ebadae12c345d7c1c252f0e41ff78c0ea56676029c5034fec2
Nov 03 16:27:35.203930 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.203899    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:27:35.204017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.203962    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:27:35.204017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.204009    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:27:35.665926 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.665885    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" event={"ID":"edac80ab-4b43-431a-b42c-c23c7232e43e","Type":"ContainerStarted","Data":"8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2"}
Nov 03 16:27:35.666429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.665981    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" event={"ID":"edac80ab-4b43-431a-b42c-c23c7232e43e","Type":"ContainerStarted","Data":"2b0ece77041f37ebadae12c345d7c1c252f0e41ff78c0ea56676029c5034fec2"}
Nov 03 16:27:35.739037 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.738994    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:35.797761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.793909    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" podStartSLOduration=1.7938885390000001 podStartE2EDuration="1.793888539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:27:35.791152748 +0000 UTC m=+941.615604020" watchObservedRunningTime="2026-11-03 16:27:35.793888539 +0000 UTC m=+941.618339814"
Nov 03 16:27:35.848623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.847957    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:35.848623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.848336    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:35.848988 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:35.848969    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:27:36.740565 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:36.740529    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-59bv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 16:27:36.740994 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:36.740588    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 16:27:37.738513 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:37.738482    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-59bv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 16:27:37.738693 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:37.738533    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 16:27:38.738744 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:38.738701    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-59bv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 16:27:38.739160 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:38.738791    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 16:27:39.739004 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:39.738974    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-59bv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 16:27:39.739395 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:39.739026    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 16:27:40.739259 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:40.739221    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-59bv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 16:27:40.739669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:40.739291    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 16:27:41.738996 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:41.738955    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-59bv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 16:27:41.739163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:41.739036    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 16:27:43.549140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:43.549105    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:43.549641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:43.549516    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:27:43.550679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:27:43.550657    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:28:03.391694 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:28:03.391644    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:28:07.517773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.517740    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"]
Nov 03 16:28:07.521308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.521287    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.577522 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.577496    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-547a38-openshift-default-dockercfg-tnnkl\""
Nov 03 16:28:07.620366 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.620326    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.620540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.620432    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.620540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.620458    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.620540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.620531    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.620794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.620560    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.620794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.620587    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.620794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.620773    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzbck\" (UniqueName: \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-kube-api-access-dzbck\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.620965 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.620820    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.621023 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.620964    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.629294 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.629268    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"]
Nov 03 16:28:07.721688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.721654    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.721688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.721691    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722025 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.721743    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722025 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.721777    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722025 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.721797    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722025 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.721820    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzbck\" (UniqueName: \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-kube-api-access-dzbck\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722025 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.721852    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722025 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.721962    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722025 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.722027    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722394 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.722172    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722394 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.722199    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722394 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.722327    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.722442    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.722713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.722648    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.724330 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.724300    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.724468 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.724452    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.782593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.782524    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.783169 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.783144    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzbck\" (UniqueName: \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-kube-api-access-dzbck\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:07.838649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:07.838616    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:08.275409 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:08.275380    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"]
Nov 03 16:28:08.278159 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:28:08.278129    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e21fcd1_f5a4_46cf_a8da_c340063f7727.slice/crio-5e911b5a591bb4ffddf91c4c0d8b99cf305271f33b18aa0134337c41361e93fb WatchSource:0}: Error finding container 5e911b5a591bb4ffddf91c4c0d8b99cf305271f33b18aa0134337c41361e93fb: Status 404 returned error can't find the container with id 5e911b5a591bb4ffddf91c4c0d8b99cf305271f33b18aa0134337c41361e93fb
Nov 03 16:28:08.280648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:08.280614    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:28:08.280764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:08.280679    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:28:08.280764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:08.280711    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:28:08.894353 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:08.894313    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" event={"ID":"5e21fcd1-f5a4-46cf-a8da-c340063f7727","Type":"ContainerStarted","Data":"ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4"}
Nov 03 16:28:08.894854 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:08.894367    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" event={"ID":"5e21fcd1-f5a4-46cf-a8da-c340063f7727","Type":"ContainerStarted","Data":"5e911b5a591bb4ffddf91c4c0d8b99cf305271f33b18aa0134337c41361e93fb"}
Nov 03 16:28:09.022381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:09.021511    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" podStartSLOduration=2.021491845 podStartE2EDuration="2.021491845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:28:09.019955697 +0000 UTC m=+974.844406968" watchObservedRunningTime="2026-11-03 16:28:09.021491845 +0000 UTC m=+974.845943117"
Nov 03 16:28:09.839794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:09.839755    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:09.841504 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:09.841486    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 16:28:09.841582 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:09.841527    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 16:28:10.839877 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:10.839843    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 16:28:10.840363 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:10.839902    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 16:28:11.839672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:11.839638    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 16:28:11.839871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:11.839702    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 16:28:12.839572 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:12.839531    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 16:28:12.840011 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:12.839603    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 16:28:13.839140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:13.839103    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 16:28:13.840122 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:13.840091    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 16:28:14.839092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:14.839052    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 16:28:14.839277 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:14.839117    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 16:28:15.842967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:15.842938    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:15.843375 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:15.843277    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:15.843942 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:15.843923    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:28:31.922203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:31.922173    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"]
Nov 03 16:28:31.926106 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:31.926087    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:31.993278 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:31.993248    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"]
Nov 03 16:28:32.027629 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.027598    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tls-certs\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.027850 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.027643    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sdpk\" (UniqueName: \"kubernetes.io/projected/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-kube-api-access-5sdpk\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.027850 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.027675    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-dshm\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.027942 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.027893    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-home\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.027981 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.027939    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-model-cache\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.028068 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.028044    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tmp-dir\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.032713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.032695    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c88d54\"/\"distinct-w2-c88d54-kserve-self-signed-certs\""
Nov 03 16:28:32.129036 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.129000    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-dshm\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.129219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.129111    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-home\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.129219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.129145    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-model-cache\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.129219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.129173    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tmp-dir\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.129372 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.129237    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tls-certs\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.129372 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.129262    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sdpk\" (UniqueName: \"kubernetes.io/projected/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-kube-api-access-5sdpk\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.129609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.129584    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-home\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.129683 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.129605    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-model-cache\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.129683 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.129623    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tmp-dir\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.131465 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.131445    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-dshm\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.131805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.131787    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tls-certs\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.192633 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.192557    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sdpk\" (UniqueName: \"kubernetes.io/projected/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-kube-api-access-5sdpk\") pod \"distinct-w2-c88d54-kserve-6db69d99f7-xkkjd\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") " pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.240701 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.240666    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:32.670531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:32.670496    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"]
Nov 03 16:28:32.673567 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:28:32.673537    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbde4efc7_f40e_4467_b076_1e2d6c47a0b0.slice/crio-511270f20b48f7a69dabb4db0befb24b199342ef477842a753a285809a5c4430 WatchSource:0}: Error finding container 511270f20b48f7a69dabb4db0befb24b199342ef477842a753a285809a5c4430: Status 404 returned error can't find the container with id 511270f20b48f7a69dabb4db0befb24b199342ef477842a753a285809a5c4430
Nov 03 16:28:33.029667 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:33.029589    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" event={"ID":"bde4efc7-f40e-4467-b076-1e2d6c47a0b0","Type":"ContainerStarted","Data":"d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8"}
Nov 03 16:28:33.029667 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:33.029633    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" event={"ID":"bde4efc7-f40e-4467-b076-1e2d6c47a0b0","Type":"ContainerStarted","Data":"511270f20b48f7a69dabb4db0befb24b199342ef477842a753a285809a5c4430"}
Nov 03 16:28:33.155065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:33.155016    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podStartSLOduration=2.155002428 podStartE2EDuration="2.155002428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:28:33.151384699 +0000 UTC m=+998.975835969" watchObservedRunningTime="2026-11-03 16:28:33.155002428 +0000 UTC m=+998.979453699"
Nov 03 16:28:33.241129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:33.241094    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:33.391141 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:28:33.391106    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:28:42.241136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:42.241094    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:42.253990 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:42.253965    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:28:43.099846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:28:43.099810    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:29:01.783850 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.783770    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"]
Nov 03 16:29:01.787150 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.787133    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.816044 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.816013    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-model-cache\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.816223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.816073    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-tmp-dir\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.816223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.816160    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-home\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.816363 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.816341    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-dshm\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.816421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.816400    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e552fb30-712d-4a82-9ddc-7af234dc9122-tls-certs\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.816483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.816452    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r7s7\" (UniqueName: \"kubernetes.io/projected/e552fb30-712d-4a82-9ddc-7af234dc9122-kube-api-access-5r7s7\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.861214 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.861176    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"]
Nov 03 16:29:01.899989 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.899958    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-6ae690\"/\"distinct-w5-6ae690-kserve-self-signed-certs\""
Nov 03 16:29:01.917107 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.917077    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-dshm\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.917275 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.917124    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e552fb30-712d-4a82-9ddc-7af234dc9122-tls-certs\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.917275 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.917158    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r7s7\" (UniqueName: \"kubernetes.io/projected/e552fb30-712d-4a82-9ddc-7af234dc9122-kube-api-access-5r7s7\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.917275 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.917254    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-model-cache\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.917451 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.917333    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-tmp-dir\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.917451 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.917390    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-home\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.917631 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.917605    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-model-cache\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.917779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.917632    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-tmp-dir\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.917779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.917681    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-home\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.920202 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.920174    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e552fb30-712d-4a82-9ddc-7af234dc9122-tls-certs\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.931031 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.920383    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-dshm\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:01.988149 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:01.988117    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r7s7\" (UniqueName: \"kubernetes.io/projected/e552fb30-712d-4a82-9ddc-7af234dc9122-kube-api-access-5r7s7\") pod \"distinct-w5-6ae690-kserve-bb4c74c99-mkt66\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") " pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:02.098869 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:02.098834    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:02.484367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:02.484342    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"]
Nov 03 16:29:02.486468 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:29:02.486441    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode552fb30_712d_4a82_9ddc_7af234dc9122.slice/crio-0f8662978c2998e78f92e66366615258d36fff74685290b20e006d5e731f6786 WatchSource:0}: Error finding container 0f8662978c2998e78f92e66366615258d36fff74685290b20e006d5e731f6786: Status 404 returned error can't find the container with id 0f8662978c2998e78f92e66366615258d36fff74685290b20e006d5e731f6786
Nov 03 16:29:03.223040 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:03.223007    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" event={"ID":"e552fb30-712d-4a82-9ddc-7af234dc9122","Type":"ContainerStarted","Data":"c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49"}
Nov 03 16:29:03.223412 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:03.223053    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" event={"ID":"e552fb30-712d-4a82-9ddc-7af234dc9122","Type":"ContainerStarted","Data":"0f8662978c2998e78f92e66366615258d36fff74685290b20e006d5e731f6786"}
Nov 03 16:29:03.346862 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:03.346808    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podStartSLOduration=2.3467947909999998 podStartE2EDuration="2.346794791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:29:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:29:03.346140716 +0000 UTC m=+1029.170591988" watchObservedRunningTime="2026-11-03 16:29:03.346794791 +0000 UTC m=+1029.171246061"
Nov 03 16:29:03.391341 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:29:03.391309    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:29:04.100418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:04.100388    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:12.099833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:12.099128    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:12.120168 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:12.120143    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:12.290514 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:12.290490    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:29:22.500111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.500063    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"]
Nov 03 16:29:22.505146 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.505125    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.610195 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.610168    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"]
Nov 03 16:29:22.611896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.611875    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-dshm\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.611986 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.611938    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tmp-dir\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.612046 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.612025    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-model-cache\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.612088 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.612073    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-home\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.612126 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.612102    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tls-certs\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.612174 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.612127    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxvkf\" (UniqueName: \"kubernetes.io/projected/ec4f033d-ff46-499b-bf4c-4712f04aaf31-kube-api-access-hxvkf\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.615080 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.615053    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-547a38\"/\"distinct-2-w6-547a38-kserve-self-signed-certs\""
Nov 03 16:29:22.713102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.713065    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-dshm\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.713373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.713134    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tmp-dir\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.713373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.713157    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-model-cache\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.713373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.713177    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-home\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.713373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.713194    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tls-certs\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.713373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.713210    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxvkf\" (UniqueName: \"kubernetes.io/projected/ec4f033d-ff46-499b-bf4c-4712f04aaf31-kube-api-access-hxvkf\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.713659 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.713586    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-model-cache\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.713659 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.713602    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-home\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.713789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.713677    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tmp-dir\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.715589 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.715567    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-dshm\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.715865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.715847    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tls-certs\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.777968 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.777903    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxvkf\" (UniqueName: \"kubernetes.io/projected/ec4f033d-ff46-499b-bf4c-4712f04aaf31-kube-api-access-hxvkf\") pod \"distinct-2-w6-547a38-kserve-8544f969d-7jbgz\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") " pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:22.817858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:22.817825    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:23.200759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:23.200717    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"]
Nov 03 16:29:23.202959 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:29:23.202937    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec4f033d_ff46_499b_bf4c_4712f04aaf31.slice/crio-fcd3786f03c0b0a2bcdf9dec6f3f362dceba1390e7c42bf4a0e500e6d9f3215f WatchSource:0}: Error finding container fcd3786f03c0b0a2bcdf9dec6f3f362dceba1390e7c42bf4a0e500e6d9f3215f: Status 404 returned error can't find the container with id fcd3786f03c0b0a2bcdf9dec6f3f362dceba1390e7c42bf4a0e500e6d9f3215f
Nov 03 16:29:23.350468 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:23.350433    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" event={"ID":"ec4f033d-ff46-499b-bf4c-4712f04aaf31","Type":"ContainerStarted","Data":"d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9"}
Nov 03 16:29:23.350468 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:23.350482    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" event={"ID":"ec4f033d-ff46-499b-bf4c-4712f04aaf31","Type":"ContainerStarted","Data":"fcd3786f03c0b0a2bcdf9dec6f3f362dceba1390e7c42bf4a0e500e6d9f3215f"}
Nov 03 16:29:23.473665 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:23.473560    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podStartSLOduration=1.473547659 podStartE2EDuration="1.473547659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:29:23.472237359 +0000 UTC m=+1049.296688631" watchObservedRunningTime="2026-11-03 16:29:23.473547659 +0000 UTC m=+1049.297998932"
Nov 03 16:29:23.818236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:23.818159    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:32.818394 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:32.818356    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:32.831121 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:32.831094    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:33.391392 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:29:33.391351    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:29:33.412240 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:33.412217    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:29:43.698891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.698851    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"]
Nov 03 16:29:43.703028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.703006    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.715520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.715492    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.715649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.715537    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.715649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.715624    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6td6\" (UniqueName: \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-kube-api-access-c6td6\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.715721 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.715659    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.715855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.715836    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.715890 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.715872    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.715977 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.715965    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.716011 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.716001    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a69b3be4-c69f-48e6-881a-436892b08ec3-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.716043 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.716025    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.764534 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.764496    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"]
Nov 03 16:29:43.817152 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.817108    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.817343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.817177    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.817343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.817213    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.817343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.817260    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6td6\" (UniqueName: \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-kube-api-access-c6td6\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.817343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.817289    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.817559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.817410    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.817559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.817451    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.817559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.817512    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.817716 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.817565    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a69b3be4-c69f-48e6-881a-436892b08ec3-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.818138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.818078    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.818428 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.818406    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a69b3be4-c69f-48e6-881a-436892b08ec3-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.818512 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.818446    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.818512 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.818496    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.818598 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.818572    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.820810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.820787    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.820810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.820803    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.878293 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.878265    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:43.878578 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:43.878552    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6td6\" (UniqueName: \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-kube-api-access-c6td6\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:44.017485 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:44.017394    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:44.454445 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:44.454418    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"]
Nov 03 16:29:44.456246 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:29:44.456220    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69b3be4_c69f_48e6_881a_436892b08ec3.slice/crio-cfb8a1d9cbb32f52d8bc3b187cd94eea6098ef46ff6c7d06e71032ab8f687e56 WatchSource:0}: Error finding container cfb8a1d9cbb32f52d8bc3b187cd94eea6098ef46ff6c7d06e71032ab8f687e56: Status 404 returned error can't find the container with id cfb8a1d9cbb32f52d8bc3b187cd94eea6098ef46ff6c7d06e71032ab8f687e56
Nov 03 16:29:44.458783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:44.458747    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:29:44.458876 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:44.458837    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:29:44.458915 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:44.458897    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:29:45.463854 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:45.463806    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" event={"ID":"a69b3be4-c69f-48e6-881a-436892b08ec3","Type":"ContainerStarted","Data":"87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c"}
Nov 03 16:29:45.464324 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:45.463869    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" event={"ID":"a69b3be4-c69f-48e6-881a-436892b08ec3","Type":"ContainerStarted","Data":"cfb8a1d9cbb32f52d8bc3b187cd94eea6098ef46ff6c7d06e71032ab8f687e56"}
Nov 03 16:29:45.700620 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:45.700550    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" podStartSLOduration=2.700529447 podStartE2EDuration="2.700529447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:29:45.643222793 +0000 UTC m=+1071.467674065" watchObservedRunningTime="2026-11-03 16:29:45.700529447 +0000 UTC m=+1071.524980719"
Nov 03 16:29:46.017970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:46.017931    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:46.019497 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:46.019470    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 16:29:46.019657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:46.019526    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 16:29:47.017759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:47.017683    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 16:29:47.018270 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:47.017778    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 16:29:48.018075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:48.018043    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 16:29:48.018464 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:48.018098    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 16:29:49.018693 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:49.018658    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 16:29:49.019144 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:49.018712    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 16:29:50.017690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:50.017654    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 16:29:50.017928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:50.017718    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 16:29:51.018435 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:51.018392    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Nov 03 16:29:51.018978 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:51.018441    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Nov 03 16:29:52.022450 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:52.022421    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:52.022955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:52.022886    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:29:52.023715 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:29:52.023691    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:30:00.193233 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.193195    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"]
Nov 03 16:30:00.198755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.198459    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"
Nov 03 16:30:00.210761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.210709    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"]
Nov 03 16:30:00.214083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.214068    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"
Nov 03 16:30:00.240694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.240667    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d79a86b3-e704-470c-aec2-627643082564-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895390-fwhtx\" (UID: \"d79a86b3-e704-470c-aec2-627643082564\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"
Nov 03 16:30:00.240845 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.240704    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zblp\" (UniqueName: \"kubernetes.io/projected/d79a86b3-e704-470c-aec2-627643082564-kube-api-access-5zblp\") pod \"maas-api-key-cleanup-29895390-fwhtx\" (UID: \"d79a86b3-e704-470c-aec2-627643082564\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"
Nov 03 16:30:00.240959 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.240940    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgngt\" (UniqueName: \"kubernetes.io/projected/ad04d796-ef55-41c8-be3c-3c93a37e419c-kube-api-access-lgngt\") pod \"maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k\" (UID: \"ad04d796-ef55-41c8-be3c-3c93a37e419c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"
Nov 03 16:30:00.241021 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.240984    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad04d796-ef55-41c8-be3c-3c93a37e419c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k\" (UID: \"ad04d796-ef55-41c8-be3c-3c93a37e419c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"
Nov 03 16:30:00.257389 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.257364    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fnmxp\""
Nov 03 16:30:00.300454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.300410    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"]
Nov 03 16:30:00.333231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.333201    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"]
Nov 03 16:30:00.342250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.342223    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgngt\" (UniqueName: \"kubernetes.io/projected/ad04d796-ef55-41c8-be3c-3c93a37e419c-kube-api-access-lgngt\") pod \"maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k\" (UID: \"ad04d796-ef55-41c8-be3c-3c93a37e419c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"
Nov 03 16:30:00.342387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.342262    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad04d796-ef55-41c8-be3c-3c93a37e419c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k\" (UID: \"ad04d796-ef55-41c8-be3c-3c93a37e419c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"
Nov 03 16:30:00.342387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.342307    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d79a86b3-e704-470c-aec2-627643082564-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895390-fwhtx\" (UID: \"d79a86b3-e704-470c-aec2-627643082564\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"
Nov 03 16:30:00.342387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.342327    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zblp\" (UniqueName: \"kubernetes.io/projected/d79a86b3-e704-470c-aec2-627643082564-kube-api-access-5zblp\") pod \"maas-api-key-cleanup-29895390-fwhtx\" (UID: \"d79a86b3-e704-470c-aec2-627643082564\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"
Nov 03 16:30:00.343029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.343008    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad04d796-ef55-41c8-be3c-3c93a37e419c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k\" (UID: \"ad04d796-ef55-41c8-be3c-3c93a37e419c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"
Nov 03 16:30:00.343246 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.343225    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d79a86b3-e704-470c-aec2-627643082564-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895390-fwhtx\" (UID: \"d79a86b3-e704-470c-aec2-627643082564\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"
Nov 03 16:30:00.407385 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.407347    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zblp\" (UniqueName: \"kubernetes.io/projected/d79a86b3-e704-470c-aec2-627643082564-kube-api-access-5zblp\") pod \"maas-api-key-cleanup-29895390-fwhtx\" (UID: \"d79a86b3-e704-470c-aec2-627643082564\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"
Nov 03 16:30:00.408198 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.408170    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgngt\" (UniqueName: \"kubernetes.io/projected/ad04d796-ef55-41c8-be3c-3c93a37e419c-kube-api-access-lgngt\") pod \"maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k\" (UID: \"ad04d796-ef55-41c8-be3c-3c93a37e419c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"
Nov 03 16:30:00.515846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.515763    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"
Nov 03 16:30:00.528080 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.528055    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"
Nov 03 16:30:00.919689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.919495    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"]
Nov 03 16:30:00.922253 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:30:00.922219    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd79a86b3_e704_470c_aec2_627643082564.slice/crio-c50afd87f426f2e1d0d80e51dae03980ff349237249cd515b80c373bc9a6e1e7 WatchSource:0}: Error finding container c50afd87f426f2e1d0d80e51dae03980ff349237249cd515b80c373bc9a6e1e7: Status 404 returned error can't find the container with id c50afd87f426f2e1d0d80e51dae03980ff349237249cd515b80c373bc9a6e1e7
Nov 03 16:30:00.924543 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.924524    5605 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 16:30:00.959971 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:00.959940    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"]
Nov 03 16:30:00.963559 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:30:00.963533    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad04d796_ef55_41c8_be3c_3c93a37e419c.slice/crio-b50a34ad180a2e23afaf891ebf494ed5ffbc304dd406bda2c79dffdc3ce2cd4e WatchSource:0}: Error finding container b50a34ad180a2e23afaf891ebf494ed5ffbc304dd406bda2c79dffdc3ce2cd4e: Status 404 returned error can't find the container with id b50a34ad180a2e23afaf891ebf494ed5ffbc304dd406bda2c79dffdc3ce2cd4e
Nov 03 16:30:01.609610 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:01.609568    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k" event={"ID":"ad04d796-ef55-41c8-be3c-3c93a37e419c","Type":"ContainerStarted","Data":"b50a34ad180a2e23afaf891ebf494ed5ffbc304dd406bda2c79dffdc3ce2cd4e"}
Nov 03 16:30:01.610989 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:01.610963    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" event={"ID":"d79a86b3-e704-470c-aec2-627643082564","Type":"ContainerStarted","Data":"c50afd87f426f2e1d0d80e51dae03980ff349237249cd515b80c373bc9a6e1e7"}
Nov 03 16:30:03.391923 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:30:03.391879    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:30:07.646141 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:07.646046    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" event={"ID":"d79a86b3-e704-470c-aec2-627643082564","Type":"ContainerStarted","Data":"3f191dac5a285c7e6e407652b159014e824476b19d5b23dc1ec77ac853aba0a1"}
Nov 03 16:30:07.647828 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:07.647795    5605 generic.go:356] "Generic (PLEG): container finished" podID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerID="ff065266fc265a54e755058c5d1956d7be4e221c4ba34f59e72fa01c12e3bcc4" exitCode=60
Nov 03 16:30:07.647971 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:07.647829    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k" event={"ID":"ad04d796-ef55-41c8-be3c-3c93a37e419c","Type":"ContainerDied","Data":"ff065266fc265a54e755058c5d1956d7be4e221c4ba34f59e72fa01c12e3bcc4"}
Nov 03 16:30:07.648075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:07.648060    5605 scope.go:122] "RemoveContainer" containerID="ff065266fc265a54e755058c5d1956d7be4e221c4ba34f59e72fa01c12e3bcc4"
Nov 03 16:30:07.768114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:07.768066    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" podStartSLOduration=1.7657829619999998 podStartE2EDuration="7.768052262s" totalImagesPullingTime="6.0022693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:30:07.766400859 +0000 UTC m=+1093.590852130" watchObservedRunningTime="2026-11-03 16:30:07.768052262 +0000 UTC m=+1093.592503533"
Nov 03 16:30:08.655077 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:08.655038    5605 generic.go:356] "Generic (PLEG): container finished" podID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerID="4158d69c91c7b45c31c27bee1287e7251cc6806155d2b75d2848d12848b345f6" exitCode=60
Nov 03 16:30:08.655492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:08.655125    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k" event={"ID":"ad04d796-ef55-41c8-be3c-3c93a37e419c","Type":"ContainerDied","Data":"4158d69c91c7b45c31c27bee1287e7251cc6806155d2b75d2848d12848b345f6"}
Nov 03 16:30:08.655492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:08.655196    5605 scope.go:122] "RemoveContainer" containerID="ff065266fc265a54e755058c5d1956d7be4e221c4ba34f59e72fa01c12e3bcc4"
Nov 03 16:30:08.655593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:08.655514    5605 scope.go:122] "RemoveContainer" containerID="4158d69c91c7b45c31c27bee1287e7251cc6806155d2b75d2848d12848b345f6"
Nov 03 16:30:08.655884 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:30:08.655856    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k_odh-ai-gateway-infra(ad04d796-ef55-41c8-be3c-3c93a37e419c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c"
Nov 03 16:30:08.657035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:08.657013    5605 generic.go:356] "Generic (PLEG): container finished" podID="d79a86b3-e704-470c-aec2-627643082564" containerID="3f191dac5a285c7e6e407652b159014e824476b19d5b23dc1ec77ac853aba0a1" exitCode=60
Nov 03 16:30:08.657124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:08.657064    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" event={"ID":"d79a86b3-e704-470c-aec2-627643082564","Type":"ContainerDied","Data":"3f191dac5a285c7e6e407652b159014e824476b19d5b23dc1ec77ac853aba0a1"}
Nov 03 16:30:08.657288 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:08.657276    5605 scope.go:122] "RemoveContainer" containerID="3f191dac5a285c7e6e407652b159014e824476b19d5b23dc1ec77ac853aba0a1"
Nov 03 16:30:09.664254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:09.664221    5605 scope.go:122] "RemoveContainer" containerID="4158d69c91c7b45c31c27bee1287e7251cc6806155d2b75d2848d12848b345f6"
Nov 03 16:30:09.664698 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:30:09.664641    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k_odh-ai-gateway-infra(ad04d796-ef55-41c8-be3c-3c93a37e419c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c"
Nov 03 16:30:09.666019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:09.665999    5605 generic.go:356] "Generic (PLEG): container finished" podID="d79a86b3-e704-470c-aec2-627643082564" containerID="23c233a3923a9cf3ec2ad5281cf706cdb2487a6cf065a3973947696ffe010540" exitCode=60
Nov 03 16:30:09.666086 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:09.666030    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" event={"ID":"d79a86b3-e704-470c-aec2-627643082564","Type":"ContainerDied","Data":"23c233a3923a9cf3ec2ad5281cf706cdb2487a6cf065a3973947696ffe010540"}
Nov 03 16:30:09.666086 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:09.666063    5605 scope.go:122] "RemoveContainer" containerID="3f191dac5a285c7e6e407652b159014e824476b19d5b23dc1ec77ac853aba0a1"
Nov 03 16:30:09.666315 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:09.666288    5605 scope.go:122] "RemoveContainer" containerID="23c233a3923a9cf3ec2ad5281cf706cdb2487a6cf065a3973947696ffe010540"
Nov 03 16:30:09.666577 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:30:09.666558    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895390-fwhtx_odh-ai-gateway-infra(d79a86b3-e704-470c-aec2-627643082564)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" podUID="d79a86b3-e704-470c-aec2-627643082564"
Nov 03 16:30:10.679856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:10.679830    5605 scope.go:122] "RemoveContainer" containerID="23c233a3923a9cf3ec2ad5281cf706cdb2487a6cf065a3973947696ffe010540"
Nov 03 16:30:10.680244 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:30:10.680090    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895390-fwhtx_odh-ai-gateway-infra(d79a86b3-e704-470c-aec2-627643082564)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" podUID="d79a86b3-e704-470c-aec2-627643082564"
Nov 03 16:30:14.771455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:14.771420    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"]
Nov 03 16:30:14.777098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:14.777074    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:14.833113 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:14.833081    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 16:30:14.885270 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:14.885233    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"]
Nov 03 16:30:14.940690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:14.940650    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-20ad82e1-jhgqm\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:14.940911 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:14.940872    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snvmm\" (UniqueName: \"kubernetes.io/projected/cf97655c-9734-415b-9c9e-a8353645d126-kube-api-access-snvmm\") pod \"maas-api-revoke-sub-20ad82e1-jhgqm\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:14.940975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:14.940911    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-xks-ca\") pod \"maas-api-revoke-sub-20ad82e1-jhgqm\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:15.042689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:15.042599    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-20ad82e1-jhgqm\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:15.042900 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:15.042750    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snvmm\" (UniqueName: \"kubernetes.io/projected/cf97655c-9734-415b-9c9e-a8353645d126-kube-api-access-snvmm\") pod \"maas-api-revoke-sub-20ad82e1-jhgqm\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:15.042900 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:15.042776    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-xks-ca\") pod \"maas-api-revoke-sub-20ad82e1-jhgqm\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:15.043320 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:15.043293    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-20ad82e1-jhgqm\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:15.045306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:15.045284    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-xks-ca\") pod \"maas-api-revoke-sub-20ad82e1-jhgqm\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:15.104565 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:15.104528    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snvmm\" (UniqueName: \"kubernetes.io/projected/cf97655c-9734-415b-9c9e-a8353645d126-kube-api-access-snvmm\") pod \"maas-api-revoke-sub-20ad82e1-jhgqm\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:15.390635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:15.390602    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:15.772001 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:15.771969    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"]
Nov 03 16:30:15.774537 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:30:15.774506    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf97655c_9734_415b_9c9e_a8353645d126.slice/crio-5335d4eb8f1e0086b4034c19ea4c4b0037b53704759c8e485309fc2731b73528 WatchSource:0}: Error finding container 5335d4eb8f1e0086b4034c19ea4c4b0037b53704759c8e485309fc2731b73528: Status 404 returned error can't find the container with id 5335d4eb8f1e0086b4034c19ea4c4b0037b53704759c8e485309fc2731b73528
Nov 03 16:30:16.712857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:16.712817    5605 generic.go:356] "Generic (PLEG): container finished" podID="cf97655c-9734-415b-9c9e-a8353645d126" containerID="6a41cb2a572d6dd9443532175cd93fd2a296f39676acd51b312b41da9587bf90" exitCode=0
Nov 03 16:30:16.713017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:16.712868    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm" event={"ID":"cf97655c-9734-415b-9c9e-a8353645d126","Type":"ContainerDied","Data":"6a41cb2a572d6dd9443532175cd93fd2a296f39676acd51b312b41da9587bf90"}
Nov 03 16:30:16.713017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:16.712909    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm" event={"ID":"cf97655c-9734-415b-9c9e-a8353645d126","Type":"ContainerStarted","Data":"5335d4eb8f1e0086b4034c19ea4c4b0037b53704759c8e485309fc2731b73528"}
Nov 03 16:30:17.860457 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:17.860421    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:17.977374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:17.977293    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf97655c-9734-415b-9c9e-a8353645d126-kube-api-access-snvmm\" (UniqueName: \"kubernetes.io/projected/cf97655c-9734-415b-9c9e-a8353645d126-kube-api-access-snvmm\") pod \"cf97655c-9734-415b-9c9e-a8353645d126\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") "
Nov 03 16:30:17.977374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:17.977346    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-openshift-ca\") pod \"cf97655c-9734-415b-9c9e-a8353645d126\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") "
Nov 03 16:30:17.977374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:17.977374    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-xks-ca\") pod \"cf97655c-9734-415b-9c9e-a8353645d126\" (UID: \"cf97655c-9734-415b-9c9e-a8353645d126\") "
Nov 03 16:30:17.977686 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:17.977663    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-openshift-ca" pod "cf97655c-9734-415b-9c9e-a8353645d126" (UID: "cf97655c-9734-415b-9c9e-a8353645d126"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:30:17.979773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:17.979716    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-xks-ca" pod "cf97655c-9734-415b-9c9e-a8353645d126" (UID: "cf97655c-9734-415b-9c9e-a8353645d126"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:30:17.979880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:17.979806    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf97655c-9734-415b-9c9e-a8353645d126-kube-api-access-snvmm" pod "cf97655c-9734-415b-9c9e-a8353645d126" (UID: "cf97655c-9734-415b-9c9e-a8353645d126"). InnerVolumeSpecName "kube-api-access-snvmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:30:18.078683 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:18.078647    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snvmm\" (UniqueName: \"kubernetes.io/projected/cf97655c-9734-415b-9c9e-a8353645d126-kube-api-access-snvmm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:30:18.078683 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:18.078678    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-openshift-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:30:18.078899 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:18.078696    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf97655c-9734-415b-9c9e-a8353645d126-maas-api-xks-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:30:18.741437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:18.740576    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"
Nov 03 16:30:18.741437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:18.740648    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm" event={"ID":"cf97655c-9734-415b-9c9e-a8353645d126","Type":"ContainerDied","Data":"5335d4eb8f1e0086b4034c19ea4c4b0037b53704759c8e485309fc2731b73528"}
Nov 03 16:30:18.741437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:18.740691    5605 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5335d4eb8f1e0086b4034c19ea4c4b0037b53704759c8e485309fc2731b73528"
Nov 03 16:30:20.301954 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:20.301923    5605 scope.go:122] "RemoveContainer" containerID="4158d69c91c7b45c31c27bee1287e7251cc6806155d2b75d2848d12848b345f6"
Nov 03 16:30:20.754115 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:20.754081    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k" event={"ID":"ad04d796-ef55-41c8-be3c-3c93a37e419c","Type":"ContainerStarted","Data":"f2f40d48e6bb5a3f4b2e96d333f80dc90cd973593fd4ce9bf05defd6d2424005"}
Nov 03 16:30:20.875418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:20.875322    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k" podStartSLOduration=15.348988245 podStartE2EDuration="20.875307956s" totalImagesPullingTime="5.526319711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:30:20.873303338 +0000 UTC m=+1106.697754610" watchObservedRunningTime="2026-11-03 16:30:20.875307956 +0000 UTC m=+1106.699759226"
Nov 03 16:30:21.761340 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:21.761306    5605 generic.go:356] "Generic (PLEG): container finished" podID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerID="f2f40d48e6bb5a3f4b2e96d333f80dc90cd973593fd4ce9bf05defd6d2424005" exitCode=60
Nov 03 16:30:21.761770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:21.761384    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k" event={"ID":"ad04d796-ef55-41c8-be3c-3c93a37e419c","Type":"ContainerDied","Data":"f2f40d48e6bb5a3f4b2e96d333f80dc90cd973593fd4ce9bf05defd6d2424005"}
Nov 03 16:30:21.761770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:21.761449    5605 scope.go:122] "RemoveContainer" containerID="4158d69c91c7b45c31c27bee1287e7251cc6806155d2b75d2848d12848b345f6"
Nov 03 16:30:21.761926 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:21.761912    5605 scope.go:122] "RemoveContainer" containerID="f2f40d48e6bb5a3f4b2e96d333f80dc90cd973593fd4ce9bf05defd6d2424005"
Nov 03 16:30:21.762220 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:30:21.762199    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k_odh-ai-gateway-infra(ad04d796-ef55-41c8-be3c-3c93a37e419c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c"
Nov 03 16:30:21.894922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:21.894890    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"]
Nov 03 16:30:22.929958 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:22.929932    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"
Nov 03 16:30:22.952130 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:22.952097    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad04d796-ef55-41c8-be3c-3c93a37e419c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad04d796-ef55-41c8-be3c-3c93a37e419c-maas-api-service-ca\") pod \"ad04d796-ef55-41c8-be3c-3c93a37e419c\" (UID: \"ad04d796-ef55-41c8-be3c-3c93a37e419c\") "
Nov 03 16:30:22.952262 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:22.952173    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad04d796-ef55-41c8-be3c-3c93a37e419c-kube-api-access-lgngt\" (UniqueName: \"kubernetes.io/projected/ad04d796-ef55-41c8-be3c-3c93a37e419c-kube-api-access-lgngt\") pod \"ad04d796-ef55-41c8-be3c-3c93a37e419c\" (UID: \"ad04d796-ef55-41c8-be3c-3c93a37e419c\") "
Nov 03 16:30:22.952472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:22.952449    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad04d796-ef55-41c8-be3c-3c93a37e419c-maas-api-service-ca" pod "ad04d796-ef55-41c8-be3c-3c93a37e419c" (UID: "ad04d796-ef55-41c8-be3c-3c93a37e419c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:30:22.954438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:22.954413    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad04d796-ef55-41c8-be3c-3c93a37e419c-kube-api-access-lgngt" pod "ad04d796-ef55-41c8-be3c-3c93a37e419c" (UID: "ad04d796-ef55-41c8-be3c-3c93a37e419c"). InnerVolumeSpecName "kube-api-access-lgngt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:30:23.053409 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:23.053334    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad04d796-ef55-41c8-be3c-3c93a37e419c-maas-api-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:30:23.053409 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:23.053364    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgngt\" (UniqueName: \"kubernetes.io/projected/ad04d796-ef55-41c8-be3c-3c93a37e419c-kube-api-access-lgngt\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:30:23.782228 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:23.782196    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"
Nov 03 16:30:23.782411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:23.782238    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k" event={"ID":"ad04d796-ef55-41c8-be3c-3c93a37e419c","Type":"ContainerDied","Data":"b50a34ad180a2e23afaf891ebf494ed5ffbc304dd406bda2c79dffdc3ce2cd4e"}
Nov 03 16:30:23.782411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:23.782304    5605 scope.go:122] "RemoveContainer" containerID="f2f40d48e6bb5a3f4b2e96d333f80dc90cd973593fd4ce9bf05defd6d2424005"
Nov 03 16:30:23.954039 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:23.954007    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"]
Nov 03 16:30:24.005073 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:24.005040    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-bcd756-29895390-88j9k"]
Nov 03 16:30:24.307416 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:24.307386    5605 scope.go:122] "RemoveContainer" containerID="23c233a3923a9cf3ec2ad5281cf706cdb2487a6cf065a3973947696ffe010540"
Nov 03 16:30:24.310544 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:24.310517    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c" path="/var/lib/kubelet/pods/ad04d796-ef55-41c8-be3c-3c93a37e419c/volumes"
Nov 03 16:30:24.795823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:24.795447    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" event={"ID":"d79a86b3-e704-470c-aec2-627643082564","Type":"ContainerStarted","Data":"f4d917bcd2f1b04aead694acb8263019fc95f3c24cc13d53ca5e1d0e63c02f4c"}
Nov 03 16:30:25.805286 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:25.805207    5605 generic.go:356] "Generic (PLEG): container finished" podID="d79a86b3-e704-470c-aec2-627643082564" containerID="f4d917bcd2f1b04aead694acb8263019fc95f3c24cc13d53ca5e1d0e63c02f4c" exitCode=60
Nov 03 16:30:25.805286 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:25.805257    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" event={"ID":"d79a86b3-e704-470c-aec2-627643082564","Type":"ContainerDied","Data":"f4d917bcd2f1b04aead694acb8263019fc95f3c24cc13d53ca5e1d0e63c02f4c"}
Nov 03 16:30:25.805799 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:25.805303    5605 scope.go:122] "RemoveContainer" containerID="23c233a3923a9cf3ec2ad5281cf706cdb2487a6cf065a3973947696ffe010540"
Nov 03 16:30:25.805799 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:25.805662    5605 scope.go:122] "RemoveContainer" containerID="f4d917bcd2f1b04aead694acb8263019fc95f3c24cc13d53ca5e1d0e63c02f4c"
Nov 03 16:30:25.806011 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:30:25.805990    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895390-fwhtx_odh-ai-gateway-infra(d79a86b3-e704-470c-aec2-627643082564)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" podUID="d79a86b3-e704-470c-aec2-627643082564"
Nov 03 16:30:25.933083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:25.933049    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"]
Nov 03 16:30:26.959310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:26.959283    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"
Nov 03 16:30:27.036450 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.036412    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d79a86b3-e704-470c-aec2-627643082564-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d79a86b3-e704-470c-aec2-627643082564-maas-api-service-ca\") pod \"d79a86b3-e704-470c-aec2-627643082564\" (UID: \"d79a86b3-e704-470c-aec2-627643082564\") "
Nov 03 16:30:27.036658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.036507    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d79a86b3-e704-470c-aec2-627643082564-kube-api-access-5zblp\" (UniqueName: \"kubernetes.io/projected/d79a86b3-e704-470c-aec2-627643082564-kube-api-access-5zblp\") pod \"d79a86b3-e704-470c-aec2-627643082564\" (UID: \"d79a86b3-e704-470c-aec2-627643082564\") "
Nov 03 16:30:27.036846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.036809    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d79a86b3-e704-470c-aec2-627643082564-maas-api-service-ca" pod "d79a86b3-e704-470c-aec2-627643082564" (UID: "d79a86b3-e704-470c-aec2-627643082564"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:30:27.036965 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.036944    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d79a86b3-e704-470c-aec2-627643082564-maas-api-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:30:27.038765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.038740    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d79a86b3-e704-470c-aec2-627643082564-kube-api-access-5zblp" pod "d79a86b3-e704-470c-aec2-627643082564" (UID: "d79a86b3-e704-470c-aec2-627643082564"). InnerVolumeSpecName "kube-api-access-5zblp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:30:27.137766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.137719    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zblp\" (UniqueName: \"kubernetes.io/projected/d79a86b3-e704-470c-aec2-627643082564-kube-api-access-5zblp\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:30:27.818368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.818341    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"
Nov 03 16:30:27.818554 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.818377    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx" event={"ID":"d79a86b3-e704-470c-aec2-627643082564","Type":"ContainerDied","Data":"c50afd87f426f2e1d0d80e51dae03980ff349237249cd515b80c373bc9a6e1e7"}
Nov 03 16:30:27.818554 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.818440    5605 scope.go:122] "RemoveContainer" containerID="f4d917bcd2f1b04aead694acb8263019fc95f3c24cc13d53ca5e1d0e63c02f4c"
Nov 03 16:30:27.990636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.990599    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"]
Nov 03 16:30:27.996826 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:27.996802    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895390-fwhtx"]
Nov 03 16:30:28.321751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:28.321109    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d79a86b3-e704-470c-aec2-627643082564" path="/var/lib/kubelet/pods/d79a86b3-e704-470c-aec2-627643082564/volumes"
Nov 03 16:30:33.391477 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:30:33.391433    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:30:44.200436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200402    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"]
Nov 03 16:30:44.200925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200853    5605 cpu_manager.go:414] "containerMap: removing container" podUID="cf97655c-9734-415b-9c9e-a8353645d126" containerName="revoke-keys"
Nov 03 16:30:44.200925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200867    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf97655c-9734-415b-9c9e-a8353645d126" containerName="revoke-keys"
Nov 03 16:30:44.200925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200881    5605 cpu_manager.go:414] "containerMap: removing container" podUID="d79a86b3-e704-470c-aec2-627643082564" containerName="cleanup"
Nov 03 16:30:44.200925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200887    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d79a86b3-e704-470c-aec2-627643082564" containerName="cleanup"
Nov 03 16:30:44.200925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200897    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerName="cleanup"
Nov 03 16:30:44.200925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200902    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerName="cleanup"
Nov 03 16:30:44.200925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200923    5605 cpu_manager.go:414] "containerMap: removing container" podUID="d79a86b3-e704-470c-aec2-627643082564" containerName="cleanup"
Nov 03 16:30:44.200925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200928    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d79a86b3-e704-470c-aec2-627643082564" containerName="cleanup"
Nov 03 16:30:44.201167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200936    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerName="cleanup"
Nov 03 16:30:44.201167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.200941    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerName="cleanup"
Nov 03 16:30:44.201167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.201014    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="d79a86b3-e704-470c-aec2-627643082564" containerName="cleanup"
Nov 03 16:30:44.201167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.201020    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerName="cleanup"
Nov 03 16:30:44.201167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.201027    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerName="cleanup"
Nov 03 16:30:44.201167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.201032    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="d79a86b3-e704-470c-aec2-627643082564" containerName="cleanup"
Nov 03 16:30:44.201167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.201039    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf97655c-9734-415b-9c9e-a8353645d126" containerName="revoke-keys"
Nov 03 16:30:44.205513 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.205494    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.261509 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.261472    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"]
Nov 03 16:30:44.310351 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.310316    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-bcd756\"/\"default-dockercfg-q6dvs\""
Nov 03 16:30:44.310514 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.310381    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-bcd756\"/\"kube-root-ca.crt\""
Nov 03 16:30:44.310514 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.310403    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-bcd756\"/\"openshift-service-ca.crt\""
Nov 03 16:30:44.311691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.311675    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-bcd756\"/\"distinct-2-w1-bcd756-kserve-self-signed-certs\""
Nov 03 16:30:44.332523 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.332500    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-dshm\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.332630 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.332569    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tmp-dir\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.332684 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.332672    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-model-cache\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.332758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.332717    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-home\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.332820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.332799    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf2zt\" (UniqueName: \"kubernetes.io/projected/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-kube-api-access-kf2zt\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.332907 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.332893    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tls-certs\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.434047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.434016    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-model-cache\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.434248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.434077    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-home\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.434248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.434094    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf2zt\" (UniqueName: \"kubernetes.io/projected/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-kube-api-access-kf2zt\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.434248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.434136    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tls-certs\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.434248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.434202    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-dshm\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.434484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.434267    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tmp-dir\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.434484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.434461    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-model-cache\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.434591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.434527    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-home\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.434665 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.434647    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tmp-dir\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.436642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.436612    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-dshm\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.436890 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.436872    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tls-certs\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.494922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.494861    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf2zt\" (UniqueName: \"kubernetes.io/projected/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-kube-api-access-kf2zt\") pod \"distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") " pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.519985 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.519959    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:44.958756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:44.958701    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"]
Nov 03 16:30:44.960304 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:30:44.960279    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6376ccbd_aea0_41b3_bd45_cdbe2f999e28.slice/crio-7984c37f33451e0e5d3105bc4b833e5ab9610295451a9dba6c7366de3d7e0333 WatchSource:0}: Error finding container 7984c37f33451e0e5d3105bc4b833e5ab9610295451a9dba6c7366de3d7e0333: Status 404 returned error can't find the container with id 7984c37f33451e0e5d3105bc4b833e5ab9610295451a9dba6c7366de3d7e0333
Nov 03 16:30:45.917768 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:45.917707    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" event={"ID":"6376ccbd-aea0-41b3-bd45-cdbe2f999e28","Type":"ContainerStarted","Data":"15aa712ca4820c14f48e19d426977909081ba713f230b595cd07447b5a952fda"}
Nov 03 16:30:45.917768 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:45.917775    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" event={"ID":"6376ccbd-aea0-41b3-bd45-cdbe2f999e28","Type":"ContainerStarted","Data":"7984c37f33451e0e5d3105bc4b833e5ab9610295451a9dba6c7366de3d7e0333"}
Nov 03 16:30:46.044158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:46.044110    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podStartSLOduration=2.044095372 podStartE2EDuration="2.044095372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:30:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:30:46.043951125 +0000 UTC m=+1131.868402395" watchObservedRunningTime="2026-11-03 16:30:46.044095372 +0000 UTC m=+1131.868546642"
Nov 03 16:30:46.520410 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:46.520372    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:54.520721 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:54.520689    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:54.533204 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:54.533176    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:30:55.026492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:30:55.026459    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:31:03.391652 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:31:03.391606    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:31:33.392421 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:31:33.392381    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:31:54.439192 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:54.439161    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:31:54.441088 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:54.441066    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:31:59.785354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.785276    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"]
Nov 03 16:31:59.785750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.785718    5605 cpu_manager.go:414] "containerMap: removing container" podUID="d79a86b3-e704-470c-aec2-627643082564" containerName="cleanup"
Nov 03 16:31:59.785750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.785746    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d79a86b3-e704-470c-aec2-627643082564" containerName="cleanup"
Nov 03 16:31:59.785831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.785778    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerName="cleanup"
Nov 03 16:31:59.785831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.785783    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerName="cleanup"
Nov 03 16:31:59.785895 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.785853    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="d79a86b3-e704-470c-aec2-627643082564" containerName="cleanup"
Nov 03 16:31:59.785895 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.785862    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad04d796-ef55-41c8-be3c-3c93a37e419c" containerName="cleanup"
Nov 03 16:31:59.788149 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.788133    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.857496 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.857466    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.857704 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.857516    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbf5v\" (UniqueName: \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-kube-api-access-fbf5v\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.857704 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.857607    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.857704 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.857655    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.858029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.857854    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.858029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.857909    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.858029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.858010    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.858173 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.858081    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.858173 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.858128    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.912216 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.912183    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"]
Nov 03 16:31:59.959240 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959205    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.959240 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959238    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.959500 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959387    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.959500 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959453    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbf5v\" (UniqueName: \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-kube-api-access-fbf5v\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.959621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959551    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.959621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959555    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.959750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959648    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.959750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959649    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.959859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959760    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.959859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959837    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.959960 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.959898    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.960101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.960078    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.960282 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.960252    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.960282 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.960312    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.962088 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.962066    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:31:59.962207 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:31:59.962189    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:32:00.020355 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:00.020315    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:32:00.020511 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:00.020355    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbf5v\" (UniqueName: \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-kube-api-access-fbf5v\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:32:00.100388 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:00.100348    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:32:00.537701 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:00.537677    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"]
Nov 03 16:32:00.539532 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:32:00.539495    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8584968f_9bed_42ab_bf5c_9d238e2ca0cd.slice/crio-013559f844766ecca8a13475c8d82cdead7bd718062da906bb0e4099ae3bf91c WatchSource:0}: Error finding container 013559f844766ecca8a13475c8d82cdead7bd718062da906bb0e4099ae3bf91c: Status 404 returned error can't find the container with id 013559f844766ecca8a13475c8d82cdead7bd718062da906bb0e4099ae3bf91c
Nov 03 16:32:00.541917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:00.541883    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:00.541992 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:00.541948    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:00.541992 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:00.541980    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:01.531533 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:01.531502    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" event={"ID":"8584968f-9bed-42ab-bf5c-9d238e2ca0cd","Type":"ContainerStarted","Data":"904878db9d16bd72b76fdcab1ff80f8ed54d7475c9e8897afadbf144cf4e6ad4"}
Nov 03 16:32:01.531950 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:01.531553    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" event={"ID":"8584968f-9bed-42ab-bf5c-9d238e2ca0cd","Type":"ContainerStarted","Data":"013559f844766ecca8a13475c8d82cdead7bd718062da906bb0e4099ae3bf91c"}
Nov 03 16:32:01.763871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:01.763813    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" podStartSLOduration=2.763800014 podStartE2EDuration="2.763800014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:31:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:32:01.761658785 +0000 UTC m=+1207.586110055" watchObservedRunningTime="2026-11-03 16:32:01.763800014 +0000 UTC m=+1207.588251284"
Nov 03 16:32:02.103166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:02.103130    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:32:02.103569 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:02.103541    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 16:32:02.103785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:02.103761    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 16:32:03.107108 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:03.107071    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 16:32:03.107521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:03.107127    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 16:32:03.417624 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:32:03.417523    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:32:04.105756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:04.104142    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 16:32:04.105756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:04.104206    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 16:32:05.106240 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:05.106186    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 16:32:05.106902 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:05.106262    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 16:32:06.105755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:06.105578    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 16:32:06.105755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:06.105639    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 16:32:07.104393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:07.104342    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 16:32:07.105077 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:07.105043    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 16:32:08.104753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:08.104068    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 16:32:08.104753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:08.104138    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 16:32:09.105849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:09.105819    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:32:09.106254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:09.106110    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:32:09.106933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:09.106904    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:32:14.586075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.586037    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"]
Nov 03 16:32:14.588912 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.588892    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.653706 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.653675    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-credential-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.653706 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.653714    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-data\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.653956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.653805    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-certs\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.653956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.653837    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlkrx\" (UniqueName: \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-kube-api-access-mlkrx\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.653956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.653941    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.654070 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.653983    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-envoy\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.654070 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.654039    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-podinfo\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.654170 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.654127    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-token\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.654236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.654218    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istiod-ca-cert\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.698356 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.698323    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"]
Nov 03 16:32:14.755356 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755328    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlkrx\" (UniqueName: \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-kube-api-access-mlkrx\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.755540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755368    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.755540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755396    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-envoy\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.755540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755440    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-podinfo\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.755540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755457    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-token\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.755540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755493    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istiod-ca-cert\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.755989 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755618    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-credential-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.755989 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755650    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-data\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.755989 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755689    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-certs\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.755989 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755804    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.756417 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.755983    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-credential-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.756417 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.756061    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-certs\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.756417 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.756130    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-data\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.756417 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.756269    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istiod-ca-cert\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.757950 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.757930    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-envoy\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.758066 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.758044    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-podinfo\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.817120 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.817081    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-token\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.817310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.817286    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlkrx\" (UniqueName: \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-kube-api-access-mlkrx\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:14.902285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:14.902204    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:15.341386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.341361    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"]
Nov 03 16:32:15.343668 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:32:15.343640    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c2fae7c_b4c0_49cf_82a0_c93f475ae57d.slice/crio-dbbe61c223c23a8dd6512f5614df1262e4df5992b862034ecd6740897a2dc817 WatchSource:0}: Error finding container dbbe61c223c23a8dd6512f5614df1262e4df5992b862034ecd6740897a2dc817: Status 404 returned error can't find the container with id dbbe61c223c23a8dd6512f5614df1262e4df5992b862034ecd6740897a2dc817
Nov 03 16:32:15.346209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.346166    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:15.346323 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.346248    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:15.346323 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.346293    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:15.567549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.567509    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"]
Nov 03 16:32:15.570559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.570532    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.636812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.636689    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"]
Nov 03 16:32:15.666573 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.666536    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-certs\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.666785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.666600    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-envoy\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.666785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.666632    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-data\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.666785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.666656    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5748322-abad-483c-ba35-10e1178e5a4d-istiod-ca-cert\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.666979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.666791    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.666979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.666821    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-istio-token\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.666979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.666856    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5748322-abad-483c-ba35-10e1178e5a4d-istio-podinfo\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.666979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.666880    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp26s\" (UniqueName: \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-kube-api-access-xp26s\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.666979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.666912    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-credential-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.668400 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.668373    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" event={"ID":"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d","Type":"ContainerStarted","Data":"f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385"}
Nov 03 16:32:15.668519 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.668446    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" event={"ID":"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d","Type":"ContainerStarted","Data":"dbbe61c223c23a8dd6512f5614df1262e4df5992b862034ecd6740897a2dc817"}
Nov 03 16:32:15.768460 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.768418    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-certs\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.768783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.768470    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-envoy\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.768783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.768495    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-data\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.768783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.768510    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5748322-abad-483c-ba35-10e1178e5a4d-istiod-ca-cert\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.768783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.768575    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.768783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.768595    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-istio-token\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.768783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.768625    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5748322-abad-483c-ba35-10e1178e5a4d-istio-podinfo\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.768783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.768642    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp26s\" (UniqueName: \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-kube-api-access-xp26s\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.768783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.768661    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-credential-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.769233 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.769086    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-data\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.770129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.769313    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-certs\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.770129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.769743    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5748322-abad-483c-ba35-10e1178e5a4d-istiod-ca-cert\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.770129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.769956    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.770331 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.770219    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-credential-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.771981 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.771962    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-envoy\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.772128 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.772107    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5748322-abad-483c-ba35-10e1178e5a4d-istio-podinfo\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.835618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.835575    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-istio-token\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.835812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.835795    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp26s\" (UniqueName: \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-kube-api-access-xp26s\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.841783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.841718    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" podStartSLOduration=1.84170582 podStartE2EDuration="1.84170582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:32:15.840222573 +0000 UTC m=+1221.664673856" watchObservedRunningTime="2026-11-03 16:32:15.84170582 +0000 UTC m=+1221.666157091"
Nov 03 16:32:15.884914 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.884853    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:15.902940 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:15.902911    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:16.342664 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:16.342633    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"]
Nov 03 16:32:16.343394 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:32:16.343370    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5748322_abad_483c_ba35_10e1178e5a4d.slice/crio-7f8b4bcb2b4ad06330d8a763ed2dc14bc1f027a12e00593acd76c49c62da75b4 WatchSource:0}: Error finding container 7f8b4bcb2b4ad06330d8a763ed2dc14bc1f027a12e00593acd76c49c62da75b4: Status 404 returned error can't find the container with id 7f8b4bcb2b4ad06330d8a763ed2dc14bc1f027a12e00593acd76c49c62da75b4
Nov 03 16:32:16.345961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:16.345928    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:16.346040 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:16.345998    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:16.346079 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:16.346053    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:16.677859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:16.677767    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" event={"ID":"f5748322-abad-483c-ba35-10e1178e5a4d","Type":"ContainerStarted","Data":"5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4"}
Nov 03 16:32:16.677859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:16.677835    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" event={"ID":"f5748322-abad-483c-ba35-10e1178e5a4d","Type":"ContainerStarted","Data":"7f8b4bcb2b4ad06330d8a763ed2dc14bc1f027a12e00593acd76c49c62da75b4"}
Nov 03 16:32:16.822100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:16.822031    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" podStartSLOduration=1.822010991 podStartE2EDuration="1.822010991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:32:16.807024437 +0000 UTC m=+1222.631475746" watchObservedRunningTime="2026-11-03 16:32:16.822010991 +0000 UTC m=+1222.646462263"
Nov 03 16:32:16.900755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:16.887834    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:16.909981 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:16.909905    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 16:32:16.909981 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:16.909967    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 16:32:17.915760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:17.908874    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 03 16:32:17.915760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:17.908940    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 03 16:32:17.927759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:17.917640    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 16:32:17.927759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:17.917700    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 16:32:18.895754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:18.892138    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 03 16:32:18.895754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:18.892202    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 03 16:32:18.905769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:18.905099    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 16:32:18.905769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:18.905184    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 16:32:19.890184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:19.890145    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 03 16:32:19.890843 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:19.890801    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 03 16:32:19.903209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:19.903171    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 16:32:19.903484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:19.903453    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 16:32:20.894753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:20.891101    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 03 16:32:20.894753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:20.891162    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 03 16:32:20.906136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:20.906100    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 16:32:20.906408 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:20.906381    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 16:32:21.898753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:21.892133    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 03 16:32:21.898753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:21.892198    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 03 16:32:21.906753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:21.903133    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 16:32:21.906753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:21.903203    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 16:32:22.894753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:22.891138    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Nov 03 16:32:22.894753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:22.891200    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Nov 03 16:32:22.980201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:22.980167    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:22.980628 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:22.980507    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:22.981538 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:22.981515    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:32:24.300875 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.300847    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:24.308495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.308472    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:24.308638 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.308539    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:32:24.478498 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.478464    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"]
Nov 03 16:32:24.481647 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.481627    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.575167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.575089    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"]
Nov 03 16:32:24.649386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.649343    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.649699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.649668    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.649848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.649763    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dce1a44-2d2b-488a-ac77-ff556bf12027-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.649848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.649833    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.649970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.649955    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.650012 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.649997    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.650048 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.650025    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shph5\" (UniqueName: \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-kube-api-access-shph5\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.650084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.650069    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.650120 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.650109    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.750839 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.750810    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.750848    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.750875    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shph5\" (UniqueName: \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-kube-api-access-shph5\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.750932    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.750970    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.750999    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.751084    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.751141    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dce1a44-2d2b-488a-ac77-ff556bf12027-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.751212    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.751300    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.751324    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.751399    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.751764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.751631    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.752376 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.752348    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dce1a44-2d2b-488a-ac77-ff556bf12027-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.753907 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.753887    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.754224 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.754203    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.814551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.814512    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:24.815142 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:24.815121    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shph5\" (UniqueName: \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-kube-api-access-shph5\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:25.094321 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:25.094287    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:25.531592 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:25.531516    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"]
Nov 03 16:32:25.565911 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:32:25.565873    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dce1a44_2d2b_488a_ac77_ff556bf12027.slice/crio-c4fa982b9ca3855566cc004f30614da2d1d72d0ef0d80aed71e781420ac4c609 WatchSource:0}: Error finding container c4fa982b9ca3855566cc004f30614da2d1d72d0ef0d80aed71e781420ac4c609: Status 404 returned error can't find the container with id c4fa982b9ca3855566cc004f30614da2d1d72d0ef0d80aed71e781420ac4c609
Nov 03 16:32:25.569141 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:25.569104    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:25.569233 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:25.569175    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:25.569233 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:25.569205    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:25.787699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:25.787600    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" event={"ID":"6dce1a44-2d2b-488a-ac77-ff556bf12027","Type":"ContainerStarted","Data":"03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7"}
Nov 03 16:32:25.787699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:25.787659    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" event={"ID":"6dce1a44-2d2b-488a-ac77-ff556bf12027","Type":"ContainerStarted","Data":"c4fa982b9ca3855566cc004f30614da2d1d72d0ef0d80aed71e781420ac4c609"}
Nov 03 16:32:25.912190 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:25.912140    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podStartSLOduration=1.9121257649999999 podStartE2EDuration="1.912125765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:32:25.911263784 +0000 UTC m=+1231.735715056" watchObservedRunningTime="2026-11-03 16:32:25.912125765 +0000 UTC m=+1231.736577035"
Nov 03 16:32:26.099152 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:26.099117    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:26.104535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:26.104500    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 03 16:32:26.104812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:26.104779    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 03 16:32:27.096753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:27.094854    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 03 16:32:27.096753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:27.094912    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 03 16:32:28.102752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:28.097063    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 03 16:32:28.102752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:28.097122    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 03 16:32:29.024753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.023818    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:29.097759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.097452    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 03 16:32:29.097759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.097550    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 03 16:32:29.263751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.262877    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:29.402875 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.402821    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:29.435951 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.435905    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:29.461868 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.461825    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:29.580132 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.580082    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:29.617863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.617818    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 03 16:32:29.633900 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.633853    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:32:29.842876 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:29.842811    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:30.031754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.025851    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:30.031754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.026282    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 16:32:30.031754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.026983    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:32:30.095219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.095129    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 03 16:32:30.095608 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.095582    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 03 16:32:30.145909 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.145865    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:30.265773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.265686    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:30.266836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.266799    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:32:30.341763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.317847    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:30.341763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.318637    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 16:32:30.523242 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.523136    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:30.540755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.525965    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:32:30.584775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.584714    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:30.585371 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.585347    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:32:30.623907 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.623852    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:30.624552 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.624530    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:32:30.630869 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.630849    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:32:30.639057 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.639018    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:32:30.642768 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:30.639581    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:32:31.099544 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.096451    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"]
Nov 03 16:32:31.131574 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.130174    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 03 16:32:31.131574 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.130252    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 03 16:32:31.146877 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.146829    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:31.254159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.239751    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"]
Nov 03 16:32:31.254159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.246697    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:32:31.254159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.246801    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:32:31.254159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.239948    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.277381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.277352    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:32:31.332754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.324204    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:31.332754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.324603    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:32:31.372269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.371821    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:32:31.404857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.404809    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"]
Nov 03 16:32:31.427930 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.427873    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-podinfo\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.428134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.427943    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-data\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.428134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.428076    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nzxf\" (UniqueName: \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-kube-api-access-5nzxf\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.428273 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.428158    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.428273 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.428196    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-token\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.428273 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.428246    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-envoy\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.428435 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.428377    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-certs\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.428495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.428448    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-credential-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.428556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.428491    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e1d654b-6a17-473c-bcb4-5933e36769c5-istiod-ca-cert\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.467397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.467367    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:32:31.468900 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.468876    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:32:31.469072 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.469061    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"]
Nov 03 16:32:31.483886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.483854    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:32:31.484132 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.468597    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.492193 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.492168    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:32:31.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.529535    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.529593    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-token\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.529627    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.529678    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-envoy\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.529705    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.529770    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.530258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.529881    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szjrn\" (UniqueName: \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-kube-api-access-szjrn\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.530258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.529930    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-certs\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.530258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.529957    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.530258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.529987    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.530258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.530096    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-credential-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.530258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.530142    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e1d654b-6a17-473c-bcb4-5933e36769c5-istiod-ca-cert\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.530258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.530232    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-podinfo\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.530614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.530286    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-data\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.530614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.530313    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.530614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.530367    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.530614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.530434    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nzxf\" (UniqueName: \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-kube-api-access-5nzxf\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.530614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.530495    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ad46f3-42de-4277-846a-8be1e943e2b9-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.537272 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.537241    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.537536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.537514    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-credential-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.537885 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.537857    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-certs\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.538499 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.538286    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-envoy\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.538973 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.538954    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-data\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.539122 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.539102    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e1d654b-6a17-473c-bcb4-5933e36769c5-istiod-ca-cert\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.542092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.542072    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-podinfo\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.621432 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.621388    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nzxf\" (UniqueName: \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-kube-api-access-5nzxf\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.626364 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.626277    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-token\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.638279 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.638241    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.638447 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.638301    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.638447 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.638372    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ad46f3-42de-4277-846a-8be1e943e2b9-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.638447 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.638415    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.638600 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.638460    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.638600 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.638501    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.638702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.638649    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjrn\" (UniqueName: \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-kube-api-access-szjrn\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.638702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.638695    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.638845 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.638742    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.640050 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.640025    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.641048 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.640348    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.641168 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.640390    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.641282 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.640909    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.641372 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.640969    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ad46f3-42de-4277-846a-8be1e943e2b9-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.643019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.642947    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.643479 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.643463    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.711779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.710484    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.711779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.711684    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szjrn\" (UniqueName: \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-kube-api-access-szjrn\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:31.727639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.727611    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:31.868665 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:31.868456    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:32.104753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:32.101216    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 03 16:32:32.104753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:32.101289    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 03 16:32:32.210387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:32.202373    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"]
Nov 03 16:32:32.220168 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:32:32.219462    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1d654b_6a17_473c_bcb4_5933e36769c5.slice/crio-f544a2f7404bb558b0d24b680042e223ba975549402e2019237f96b2fdd3fe81 WatchSource:0}: Error finding container f544a2f7404bb558b0d24b680042e223ba975549402e2019237f96b2fdd3fe81: Status 404 returned error can't find the container with id f544a2f7404bb558b0d24b680042e223ba975549402e2019237f96b2fdd3fe81
Nov 03 16:32:32.222868 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:32.222824    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:32.222983 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:32.222918    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:32.222983 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:32.222949    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:32.320746 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:32:32.319545    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ad46f3_42de_4277_846a_8be1e943e2b9.slice/crio-6eaa1d5a38427596565e08b2ba0912c50a70d5004be1c9869b16a6af1daf34bd WatchSource:0}: Error finding container 6eaa1d5a38427596565e08b2ba0912c50a70d5004be1c9869b16a6af1daf34bd: Status 404 returned error can't find the container with id 6eaa1d5a38427596565e08b2ba0912c50a70d5004be1c9869b16a6af1daf34bd
Nov 03 16:32:32.322793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:32.322713    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"]
Nov 03 16:32:32.324408 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:32.324371    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:32.324517 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:32.324452    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:32.324517 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:32.324497    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:33.095148 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.095113    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Nov 03 16:32:33.095297 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.095169    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Nov 03 16:32:33.198811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.198764    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" event={"ID":"50ad46f3-42de-4277-846a-8be1e943e2b9","Type":"ContainerStarted","Data":"f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b"}
Nov 03 16:32:33.199008 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.198834    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" event={"ID":"50ad46f3-42de-4277-846a-8be1e943e2b9","Type":"ContainerStarted","Data":"6eaa1d5a38427596565e08b2ba0912c50a70d5004be1c9869b16a6af1daf34bd"}
Nov 03 16:32:33.200821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.200796    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" event={"ID":"4e1d654b-6a17-473c-bcb4-5933e36769c5","Type":"ContainerStarted","Data":"aef0d03426c583f3bc4bac6ddf427a0b8e147891de38b3b8eec54f93e5b4561d"}
Nov 03 16:32:33.200949 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.200838    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" event={"ID":"4e1d654b-6a17-473c-bcb4-5933e36769c5","Type":"ContainerStarted","Data":"f544a2f7404bb558b0d24b680042e223ba975549402e2019237f96b2fdd3fe81"}
Nov 03 16:32:33.342859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.335380    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" podStartSLOduration=2.335360366 podStartE2EDuration="2.335360366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:32:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:32:33.333195924 +0000 UTC m=+1239.157647196" watchObservedRunningTime="2026-11-03 16:32:33.335360366 +0000 UTC m=+1239.159811638"
Nov 03 16:32:33.446624 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:32:33.446533    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:32:33.467598 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.467549    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" podStartSLOduration=3.46753314 podStartE2EDuration="3.46753314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:32:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:32:33.465417876 +0000 UTC m=+1239.289869183" watchObservedRunningTime="2026-11-03 16:32:33.46753314 +0000 UTC m=+1239.291984412"
Nov 03 16:32:33.733810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.732807    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:33.733810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.733033    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 16:32:33.733810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.733084    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 16:32:33.869550 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.869516    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:33.870943 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.870907    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 16:32:33.871095 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:33.870967    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 16:32:34.737762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:34.734789    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 16:32:34.737762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:34.734849    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 16:32:34.869940 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:34.869907    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 16:32:34.870137 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:34.869964    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 16:32:35.121886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:35.121845    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:32:35.122200 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:35.122170    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:35.734692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:35.732023    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 16:32:35.734692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:35.732088    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 16:32:35.869088 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:35.869056    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 16:32:35.869455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:35.869102    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 16:32:36.172640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:36.171805    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:32:36.172640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:36.171858    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:32:36.429804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:36.429072    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:36.432753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:36.430521    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:36.452007 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:36.451977    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:32:36.732395 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:36.732287    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 16:32:36.732564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:36.732393    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 16:32:36.870136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:36.870095    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 16:32:36.870514 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:36.870159    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 16:32:37.730646 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:37.730605    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 16:32:37.730958 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:37.730926    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 16:32:37.869393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:37.869362    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 16:32:37.869590 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:37.869425    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 16:32:38.728193 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:38.728151    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 16:32:38.728854 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:38.728814    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 16:32:38.869164 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:38.869132    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 16:32:38.869340 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:38.869182    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 16:32:40.168595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:40.168490    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:40.175256 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:40.170538    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:40.179047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:40.178762    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:32:40.191783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:40.191716    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:40.192363 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:40.192336    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:40.193467 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:40.193449    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:32:45.493498 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.493468    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"]
Nov 03 16:32:45.496611 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.496591    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.550857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.550824    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922c42d0-8663-4e36-99ad-4f8326b16823-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.551024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.550963    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.551024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.551000    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.551147 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.551108    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.551183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.551145    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grsr6\" (UniqueName: \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-kube-api-access-grsr6\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.551183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.551176    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.551303 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.551289    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.551343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.551328    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.551390 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.551347    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922c42d0-8663-4e36-99ad-4f8326b16823-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.555921 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.555888    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"]
Nov 03 16:32:45.652535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.652503    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.652711 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.652543    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.652711 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.652598    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.652711 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.652620    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grsr6\" (UniqueName: \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-kube-api-access-grsr6\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.652711 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.652653    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.653003 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.652766    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.653003 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.652819    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.653003 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.652848    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922c42d0-8663-4e36-99ad-4f8326b16823-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.653003 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.652968    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922c42d0-8663-4e36-99ad-4f8326b16823-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.653194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.653030    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.653194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.653079    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.653194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.653104    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.653194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.653182    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.653674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.653655    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922c42d0-8663-4e36-99ad-4f8326b16823-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.655292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.655268    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.655739 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.655712    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922c42d0-8663-4e36-99ad-4f8326b16823-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.713334 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.713303    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.713497 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.713475    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grsr6\" (UniqueName: \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-kube-api-access-grsr6\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:45.811602 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:45.811512    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:46.182109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.182077    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"]
Nov 03 16:32:46.185772 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.185751    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.246940 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.246907    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"]
Nov 03 16:32:46.261388 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.261354    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-data\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.261559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.261403    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj8cp\" (UniqueName: \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-kube-api-access-dj8cp\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.261760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.261640    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-token\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.261760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.261702    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-envoy\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.261926 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.261792    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-podinfo\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.261926 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.261856    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-certs\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.261926 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.261916    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6d14b41-06d9-4fce-92fd-de9a147d3316-istiod-ca-cert\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.262109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.262022    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.262109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.262073    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-credential-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.290521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.290484    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:46.290633 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.290593    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:46.290691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.290641    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:46.297619 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.297597    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"]
Nov 03 16:32:46.327933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.327902    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" event={"ID":"922c42d0-8663-4e36-99ad-4f8326b16823","Type":"ContainerStarted","Data":"9e312ffd4ecd7136a3690e14e60089b96feb9b2aba5948c63ccda4b1dfc13625"}
Nov 03 16:32:46.363706 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.363674    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-podinfo\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.363841 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.363748    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-certs\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.363841 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.363772    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6d14b41-06d9-4fce-92fd-de9a147d3316-istiod-ca-cert\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.363841 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.363834    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.363950 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.363877    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-credential-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.364112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.364084    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-data\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.364199 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.364140    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj8cp\" (UniqueName: \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-kube-api-access-dj8cp\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.364199 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.364168    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-credential-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.364310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.364197    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-certs\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.364310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.364282    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-token\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.364408 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.364328    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-envoy\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.364678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.364526    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6d14b41-06d9-4fce-92fd-de9a147d3316-istiod-ca-cert\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.364678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.364564    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-data\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.364678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.364585    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-socket\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.366624 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.366590    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-podinfo\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.366624 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.366614    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-envoy\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.425107 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.425070    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-token\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.425279 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.425254    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj8cp\" (UniqueName: \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-kube-api-access-dj8cp\") pod \"e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") " pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.499163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.499089    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:46.945852 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.945822    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"]
Nov 03 16:32:46.947004 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:32:46.946978    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d14b41_06d9_4fce_92fd_de9a147d3316.slice/crio-5b1d794d4f4841d5404fb9a20a177d603f8017f621baa45630ce369792da1eb1 WatchSource:0}: Error finding container 5b1d794d4f4841d5404fb9a20a177d603f8017f621baa45630ce369792da1eb1: Status 404 returned error can't find the container with id 5b1d794d4f4841d5404fb9a20a177d603f8017f621baa45630ce369792da1eb1
Nov 03 16:32:46.949779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.949708    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:46.949878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.949795    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:46.949878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:46.949827    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:32:47.335610 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:47.335575    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" event={"ID":"f6d14b41-06d9-4fce-92fd-de9a147d3316","Type":"ContainerStarted","Data":"bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1"}
Nov 03 16:32:47.335823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:47.335639    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" event={"ID":"f6d14b41-06d9-4fce-92fd-de9a147d3316","Type":"ContainerStarted","Data":"5b1d794d4f4841d5404fb9a20a177d603f8017f621baa45630ce369792da1eb1"}
Nov 03 16:32:47.341371 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:47.341334    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" event={"ID":"922c42d0-8663-4e36-99ad-4f8326b16823","Type":"ContainerStarted","Data":"5e6c091adb23699d176761af10ad19a254dfa027bb75aef25e85e6460e73cf46"}
Nov 03 16:32:47.463582 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:47.463521    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" podStartSLOduration=1.463506623 podStartE2EDuration="1.463506623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:32:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:32:47.461980335 +0000 UTC m=+1253.286431618" watchObservedRunningTime="2026-11-03 16:32:47.463506623 +0000 UTC m=+1253.287957893"
Nov 03 16:32:47.507783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:47.503027    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:47.814213 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:47.814121    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:47.825101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:47.825064    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 16:32:47.825398 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:47.825358    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 16:32:48.521754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:48.504487    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 16:32:48.521754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:48.504545    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 16:32:48.815245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:48.815150    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 16:32:48.816575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:48.816542    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 16:32:49.500750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:49.500206    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 16:32:49.500750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:49.500282    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 16:32:49.814812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:49.813148    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 16:32:49.814812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:49.813220    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 16:32:50.519753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:50.502056    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 16:32:50.519753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:50.502132    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 16:32:50.813813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:50.812160    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 16:32:50.813813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:50.812236    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 16:32:51.510752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:51.509856    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 16:32:51.510752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:51.509919    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 16:32:51.828818 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:51.828384    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 16:32:51.828818 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:51.828466    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 16:32:51.945563 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:51.945502    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" podStartSLOduration=6.945486236 podStartE2EDuration="6.945486236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:32:47.628049737 +0000 UTC m=+1253.452501008" watchObservedRunningTime="2026-11-03 16:32:51.945486236 +0000 UTC m=+1257.769937507"
Nov 03 16:32:51.948032 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:51.948006    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"]
Nov 03 16:32:51.948240 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:51.948217    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy" containerID="cri-o://267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0" gracePeriod=30
Nov 03 16:32:52.018753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:52.018433    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"]
Nov 03 16:32:52.018753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:52.018678    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerName="istio-proxy" containerID="cri-o://8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2" gracePeriod=30
Nov 03 16:32:52.503111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:52.503075    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 16:32:52.503389 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:52.503363    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 16:32:52.812324 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:52.812238    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 16:32:52.812324 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:52.812294    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 16:32:53.499812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:53.499767    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 16:32:53.500019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:53.499838    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 16:32:53.818058 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:53.817975    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:53.818473 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:53.818403    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:53.819300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:53.819281    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:32:54.504429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:54.504399    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:54.504699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:54.504678    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:54.505466 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:54.505444    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:32:57.236314 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.236288    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:32:57.316865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.316842    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:32:57.330102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330082    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-token\" (UniqueName: \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-token\") pod \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") "
Nov 03 16:32:57.330220 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330203    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-data\") pod \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") "
Nov 03 16:32:57.330274 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330255    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-socket\") pod \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") "
Nov 03 16:32:57.330321 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330308    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-certs\") pod \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") "
Nov 03 16:32:57.330372 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330351    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istiod-ca-cert\") pod \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") "
Nov 03 16:32:57.330428 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330384    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-kube-api-access-sj9xw\" (UniqueName: \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-kube-api-access-sj9xw\") pod \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") "
Nov 03 16:32:57.330477 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330429    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-envoy\") pod \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") "
Nov 03 16:32:57.330538 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330486    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-data" pod "0b7e3f2a-1dff-4dad-8953-08b9337c6140" (UID: "0b7e3f2a-1dff-4dad-8953-08b9337c6140"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:32:57.330538 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330492    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-credential-socket\") pod \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") "
Nov 03 16:32:57.330640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330522    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-socket" pod "0b7e3f2a-1dff-4dad-8953-08b9337c6140" (UID: "0b7e3f2a-1dff-4dad-8953-08b9337c6140"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:32:57.330640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330611    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-podinfo\") pod \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\" (UID: \"0b7e3f2a-1dff-4dad-8953-08b9337c6140\") "
Nov 03 16:32:57.330640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330593    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-certs" pod "0b7e3f2a-1dff-4dad-8953-08b9337c6140" (UID: "0b7e3f2a-1dff-4dad-8953-08b9337c6140"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:32:57.330804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330688    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-credential-socket" pod "0b7e3f2a-1dff-4dad-8953-08b9337c6140" (UID: "0b7e3f2a-1dff-4dad-8953-08b9337c6140"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:32:57.330804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.330779    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istiod-ca-cert" pod "0b7e3f2a-1dff-4dad-8953-08b9337c6140" (UID: "0b7e3f2a-1dff-4dad-8953-08b9337c6140"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:32:57.331458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.331430    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.331573 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.331462    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.331573 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.331479    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.331573 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.331494    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.331573 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.331509    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.332618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.332594    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-token" pod "0b7e3f2a-1dff-4dad-8953-08b9337c6140" (UID: "0b7e3f2a-1dff-4dad-8953-08b9337c6140"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:32:57.332700 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.332624    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-kube-api-access-sj9xw" pod "0b7e3f2a-1dff-4dad-8953-08b9337c6140" (UID: "0b7e3f2a-1dff-4dad-8953-08b9337c6140"). InnerVolumeSpecName "kube-api-access-sj9xw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:32:57.332700 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.332664    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-envoy" pod "0b7e3f2a-1dff-4dad-8953-08b9337c6140" (UID: "0b7e3f2a-1dff-4dad-8953-08b9337c6140"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:32:57.332803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.332772    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-podinfo" pod "0b7e3f2a-1dff-4dad-8953-08b9337c6140" (UID: "0b7e3f2a-1dff-4dad-8953-08b9337c6140"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:32:57.432979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.432948    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-podinfo\") pod \"edac80ab-4b43-431a-b42c-c23c7232e43e\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") "
Nov 03 16:32:57.433143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433033    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-certs\") pod \"edac80ab-4b43-431a-b42c-c23c7232e43e\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") "
Nov 03 16:32:57.433143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433063    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-envoy\") pod \"edac80ab-4b43-431a-b42c-c23c7232e43e\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") "
Nov 03 16:32:57.433143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433104    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-token\" (UniqueName: \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-token\") pod \"edac80ab-4b43-431a-b42c-c23c7232e43e\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") "
Nov 03 16:32:57.433143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433128    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-kube-api-access-h4nms\" (UniqueName: \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-kube-api-access-h4nms\") pod \"edac80ab-4b43-431a-b42c-c23c7232e43e\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") "
Nov 03 16:32:57.433360 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433171    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-credential-socket\") pod \"edac80ab-4b43-431a-b42c-c23c7232e43e\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") "
Nov 03 16:32:57.433360 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433226    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-socket\") pod \"edac80ab-4b43-431a-b42c-c23c7232e43e\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") "
Nov 03 16:32:57.433910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433355    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edac80ab-4b43-431a-b42c-c23c7232e43e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edac80ab-4b43-431a-b42c-c23c7232e43e-istiod-ca-cert\") pod \"edac80ab-4b43-431a-b42c-c23c7232e43e\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") "
Nov 03 16:32:57.433910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433398    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-certs" pod "edac80ab-4b43-431a-b42c-c23c7232e43e" (UID: "edac80ab-4b43-431a-b42c-c23c7232e43e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:32:57.433910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433425    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-data\") pod \"edac80ab-4b43-431a-b42c-c23c7232e43e\" (UID: \"edac80ab-4b43-431a-b42c-c23c7232e43e\") "
Nov 03 16:32:57.433910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433594    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-credential-socket" pod "edac80ab-4b43-431a-b42c-c23c7232e43e" (UID: "edac80ab-4b43-431a-b42c-c23c7232e43e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:32:57.433910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433650    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-socket" pod "edac80ab-4b43-431a-b42c-c23c7232e43e" (UID: "edac80ab-4b43-431a-b42c-c23c7232e43e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:32:57.433910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433800    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-data" pod "edac80ab-4b43-431a-b42c-c23c7232e43e" (UID: "edac80ab-4b43-431a-b42c-c23c7232e43e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:32:57.434209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.433965    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edac80ab-4b43-431a-b42c-c23c7232e43e-istiod-ca-cert" pod "edac80ab-4b43-431a-b42c-c23c7232e43e" (UID: "edac80ab-4b43-431a-b42c-c23c7232e43e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:32:57.434209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.434082    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.434209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.434102    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj9xw\" (UniqueName: \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-kube-api-access-sj9xw\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.434209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.434120    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.434209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.434136    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.434209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.434150    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.434209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.434163    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.434209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.434177    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.434209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.434192    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b7e3f2a-1dff-4dad-8953-08b9337c6140-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.434209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.434206    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edac80ab-4b43-431a-b42c-c23c7232e43e-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.436027 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.435979    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-podinfo" pod "edac80ab-4b43-431a-b42c-c23c7232e43e" (UID: "edac80ab-4b43-431a-b42c-c23c7232e43e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:32:57.436283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.436257    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-kube-api-access-h4nms" pod "edac80ab-4b43-431a-b42c-c23c7232e43e" (UID: "edac80ab-4b43-431a-b42c-c23c7232e43e"). InnerVolumeSpecName "kube-api-access-h4nms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:32:57.436373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.436315    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-token" pod "edac80ab-4b43-431a-b42c-c23c7232e43e" (UID: "edac80ab-4b43-431a-b42c-c23c7232e43e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:32:57.436635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.436619    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-envoy" pod "edac80ab-4b43-431a-b42c-c23c7232e43e" (UID: "edac80ab-4b43-431a-b42c-c23c7232e43e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:32:57.464180 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.464146    5605 generic.go:356] "Generic (PLEG): container finished" podID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerID="8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2" exitCode=0
Nov 03 16:32:57.464397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.464192    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"
Nov 03 16:32:57.464397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.464224    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" event={"ID":"edac80ab-4b43-431a-b42c-c23c7232e43e","Type":"ContainerDied","Data":"8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2"}
Nov 03 16:32:57.464397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.464286    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8" event={"ID":"edac80ab-4b43-431a-b42c-c23c7232e43e","Type":"ContainerDied","Data":"2b0ece77041f37ebadae12c345d7c1c252f0e41ff78c0ea56676029c5034fec2"}
Nov 03 16:32:57.464397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.464317    5605 scope.go:122] "RemoveContainer" containerID="8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2"
Nov 03 16:32:57.474807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.472022    5605 generic.go:356] "Generic (PLEG): container finished" podID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerID="267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0" exitCode=0
Nov 03 16:32:57.474807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.472117    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" event={"ID":"0b7e3f2a-1dff-4dad-8953-08b9337c6140","Type":"ContainerDied","Data":"267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0"}
Nov 03 16:32:57.474807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.472166    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq" event={"ID":"0b7e3f2a-1dff-4dad-8953-08b9337c6140","Type":"ContainerDied","Data":"932027a629293d6e5562d2d7c896bae27a9f45862ea17be1b1313fcd232f7da7"}
Nov 03 16:32:57.474807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.472250    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"
Nov 03 16:32:57.478317 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.478299    5605 scope.go:122] "RemoveContainer" containerID="8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2"
Nov 03 16:32:57.478802 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:32:57.478643    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2\": container with ID starting with 8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2 not found: ID does not exist" containerID="8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2"
Nov 03 16:32:57.478802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.478677    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2"} err="failed to get container status \"8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2\": rpc error: code = NotFound desc = could not find container \"8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2\": container with ID starting with 8b92fab33da2ff21fcde37ddb64c515701b5429fec654e1a9633805054b791b2 not found: ID does not exist"
Nov 03 16:32:57.478802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.478702    5605 scope.go:122] "RemoveContainer" containerID="267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0"
Nov 03 16:32:57.491090 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.491054    5605 scope.go:122] "RemoveContainer" containerID="267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0"
Nov 03 16:32:57.491369 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:32:57.491352    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0\": container with ID starting with 267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0 not found: ID does not exist" containerID="267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0"
Nov 03 16:32:57.491418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.491378    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0"} err="failed to get container status \"267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0\": rpc error: code = NotFound desc = could not find container \"267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0\": container with ID starting with 267b530b12bf52a1d1d4165759c8d141acbd77c554ec493614ec2bbdb0d69bb0 not found: ID does not exist"
Nov 03 16:32:57.535892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.535856    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.535892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.535889    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.535892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.535902    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.536140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.535914    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4nms\" (UniqueName: \"kubernetes.io/projected/edac80ab-4b43-431a-b42c-c23c7232e43e-kube-api-access-h4nms\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:32:57.662948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.662913    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"]
Nov 03 16:32:57.667976 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.667949    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-b2zrq"]
Nov 03 16:32:57.782612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.782527    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"]
Nov 03 16:32:57.791234 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:57.791205    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-59bv8"]
Nov 03 16:32:58.309587 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:58.309557    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" path="/var/lib/kubelet/pods/0b7e3f2a-1dff-4dad-8953-08b9337c6140/volumes"
Nov 03 16:32:58.310103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:32:58.310087    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" path="/var/lib/kubelet/pods/edac80ab-4b43-431a-b42c-c23c7232e43e/volumes"
Nov 03 16:33:00.300980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.300943    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"]
Nov 03 16:33:00.301761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.301700    5605 cpu_manager.go:414] "containerMap: removing container" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerName="istio-proxy"
Nov 03 16:33:00.301761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.301742    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerName="istio-proxy"
Nov 03 16:33:00.301937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.301785    5605 cpu_manager.go:414] "containerMap: removing container" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy"
Nov 03 16:33:00.301937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.301793    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy"
Nov 03 16:33:00.301937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.301935    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="edac80ab-4b43-431a-b42c-c23c7232e43e" containerName="istio-proxy"
Nov 03 16:33:00.302086 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.301953    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b7e3f2a-1dff-4dad-8953-08b9337c6140" containerName="istio-proxy"
Nov 03 16:33:00.307133 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.307106    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.381580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.381541    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvw9j\" (UniqueName: \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-kube-api-access-lvw9j\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.381792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.381626    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.381792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.381660    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.381792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.381719    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.381792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.381776    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231bee41-ee7a-4b55-ad4c-4b39a0104869-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.382020 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.381803    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.382020 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.381858    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.382020 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.381912    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.382020 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.381930    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.385283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.385253    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"]
Nov 03 16:33:00.483217 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.483158    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvw9j\" (UniqueName: \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-kube-api-access-lvw9j\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.483411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.483255    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.483411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.483288    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.483411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.483345    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.483411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.483393    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231bee41-ee7a-4b55-ad4c-4b39a0104869-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.483654 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.483435    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.483654 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.483509    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.483654 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.483587    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.483654 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.483614    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.483856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.483687    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-credential-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.484140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.484117    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-certs\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.484140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.484132    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-data\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.484277 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.484174    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231bee41-ee7a-4b55-ad4c-4b39a0104869-istiod-ca-cert\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.484277 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.484177    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-socket\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.486179 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.486152    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-envoy\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.486414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.486398    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-podinfo\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.544433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.544391    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-token\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.544580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.544499    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvw9j\" (UniqueName: \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-kube-api-access-lvw9j\") pod \"e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") " pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.619935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.619902    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"]
Nov 03 16:33:00.621177 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.621151    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:00.625752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.625475    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.687979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.686323    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.687979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.686367    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.687979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.686427    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.687979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.686471    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.687979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.686629    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv6wm\" (UniqueName: \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-kube-api-access-nv6wm\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.687979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.686657    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.687979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.686715    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"]
Nov 03 16:33:00.687979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.686786    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.687979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.686811    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.687979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.686836    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.788408 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.788372    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv6wm\" (UniqueName: \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-kube-api-access-nv6wm\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.788408 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.788412    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.788658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.788476    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.788658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.788490    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.788658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.788600    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.788871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.788697    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.788871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.788753    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.788871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.788816    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.788871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.788867    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.789124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.789099    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.789188 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.789125    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.789253 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.789229    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.789396 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.789376    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.789435 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.789419    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.790968 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.790947    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.791191 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.791174    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.857365 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.857337    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.857508 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.857485    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv6wm\" (UniqueName: \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-kube-api-access-nv6wm\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:00.958185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:00.958107    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:01.063133 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:33:01.063093    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod231bee41_ee7a_4b55_ad4c_4b39a0104869.slice/crio-368aa2e9428aca10cd686d50d041b90afa581d1db23a45e82f1afab14f27ebf1 WatchSource:0}: Error finding container 368aa2e9428aca10cd686d50d041b90afa581d1db23a45e82f1afab14f27ebf1: Status 404 returned error can't find the container with id 368aa2e9428aca10cd686d50d041b90afa581d1db23a45e82f1afab14f27ebf1
Nov 03 16:33:01.063928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.063901    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"]
Nov 03 16:33:01.065956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.065921    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:01.066042 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.065989    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:01.066042 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.066019    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:01.342410 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.342377    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"]
Nov 03 16:33:01.347867 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:33:01.347829    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc89ea04b_5366_4ba9_b3a3_30926721c6fb.slice/crio-8d379f2113efefda8d3755f2cc7b66f3ead29ead1230de17ba3bd9836be658a2 WatchSource:0}: Error finding container 8d379f2113efefda8d3755f2cc7b66f3ead29ead1230de17ba3bd9836be658a2: Status 404 returned error can't find the container with id 8d379f2113efefda8d3755f2cc7b66f3ead29ead1230de17ba3bd9836be658a2
Nov 03 16:33:01.350964 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.350928    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:01.351120 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.350996    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:01.351120 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.351024    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:01.504132 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.504088    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" event={"ID":"c89ea04b-5366-4ba9-b3a3-30926721c6fb","Type":"ContainerStarted","Data":"7554893895a7c1e24e2bba1aff977813e01e0bddb92bc4da89e6440a7f1fadd8"}
Nov 03 16:33:01.504327 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.504155    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" event={"ID":"c89ea04b-5366-4ba9-b3a3-30926721c6fb","Type":"ContainerStarted","Data":"8d379f2113efefda8d3755f2cc7b66f3ead29ead1230de17ba3bd9836be658a2"}
Nov 03 16:33:01.507745 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.506293    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" event={"ID":"231bee41-ee7a-4b55-ad4c-4b39a0104869","Type":"ContainerStarted","Data":"7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef"}
Nov 03 16:33:01.507745 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.506351    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" event={"ID":"231bee41-ee7a-4b55-ad4c-4b39a0104869","Type":"ContainerStarted","Data":"368aa2e9428aca10cd686d50d041b90afa581d1db23a45e82f1afab14f27ebf1"}
Nov 03 16:33:01.621779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.621681    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:01.633585 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.633529    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podStartSLOduration=1.633514846 podStartE2EDuration="1.633514846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:33:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:33:01.630854247 +0000 UTC m=+1267.455305518" watchObservedRunningTime="2026-11-03 16:33:01.633514846 +0000 UTC m=+1267.457966169"
Nov 03 16:33:01.764401 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.764347    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podStartSLOduration=1.76433219 podStartE2EDuration="1.76433219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:33:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:33:01.76044966 +0000 UTC m=+1267.584900932" watchObservedRunningTime="2026-11-03 16:33:01.76433219 +0000 UTC m=+1267.588783461"
Nov 03 16:33:01.959313 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:01.959201    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:02.635751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:02.635363    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 16:33:02.635751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:02.635417    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 16:33:02.973939 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:02.972951    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 16:33:02.973939 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:02.973033    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 16:33:03.613891 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:33:03.609308    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:33:03.622193 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:03.622154    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 16:33:03.622493 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:03.622449    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 16:33:03.962818 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:03.961184    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 16:33:03.962818 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:03.961264    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 16:33:04.306810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:04.302911    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 16:33:04.306810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:04.303379    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 16:33:04.403757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:04.391968    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:04.468897 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:04.468842    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:04.483887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:04.483841    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:04.511763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:04.509353    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:04.623351 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:04.623215    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 16:33:04.623351 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:04.623288    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 16:33:04.858130 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:04.858086    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 16:33:04.979620 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:04.979521    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 16:33:05.013851 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.013807    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 16:33:05.079947 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.079901    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:05.119045 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.118985    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 03 16:33:05.315922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.315816    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:05.316863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.316837    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:33:05.321987 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.321949    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 16:33:05.322548 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.322527    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:33:05.398751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.397878    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:33:05.398751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.398410    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:33:05.484918 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.484873    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 03 16:33:05.485548 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.485525    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:33:05.487045 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.487011    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:05.488937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.488119    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:33:05.511774 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.511702    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 03 16:33:05.512174 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.512136    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:33:05.512953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.512923    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:05.539122 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.537207    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:05.664754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.663903    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:05.732377 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.732339    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 16:33:05.759008 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.758960    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 16:33:05.779039 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.778901    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:05.897634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.897588    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 16:33:05.899179 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.899153    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:33:05.961332 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.961237    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 16:33:05.961584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:05.961554    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 16:33:06.094937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.094887    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:06.117925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.117879    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:33:06.133487 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.130293    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:06.133928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.133904    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:33:06.465758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.462551    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:33:06.537789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.491918    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:06.537789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.492497    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 16:33:06.537789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.525510    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 03 16:33:06.537789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.526965    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:06.537789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.527024    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:06.537789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.527102    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 16:33:06.665455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.622193    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 03 16:33:06.665455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.623881    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 16:33:06.889817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.831000    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 03 16:33:07.057781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.978035    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 16:33:07.057781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.057019    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 16:33:07.057781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.979158    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 16:33:07.057781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.057088    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 16:33:07.057781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:06.980965    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:07.384558 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.368644    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:07.384558 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.369028    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:07.384558 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.382147    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:33:07.384558 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.382226    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:33:07.502915 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.502867    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:07.510760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.503522    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 03 16:33:07.544834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.517954    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:33:07.545690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.545660    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 03 16:33:07.547990 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.545595    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:07.549420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.549379    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:07.569012 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.568870    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:07.569012 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.568938    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:07.662811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.630334    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 16:33:07.663091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.663042    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 16:33:07.663250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.632503    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 03 16:33:07.700755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.632466    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:07.701370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.646704    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/health\": context deadline exceeded"
Nov 03 16:33:07.983829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.967187    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 16:33:07.984140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:07.984100    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 16:33:08.059752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.059007    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:08.059752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.059369    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 03 16:33:08.405753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.369289    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 03 16:33:08.405753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.369753    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 16:33:08.513754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.506887    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:08.521754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.517807    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 03 16:33:08.554758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.540745    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:08.562758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.555248    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:08.594658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.556837    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:08.637755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.634883    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 16:33:08.637755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.635008    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 16:33:08.680811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.679879    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:08.686766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.682586    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:08.729753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.706782    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:08.966472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.966283    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 16:33:08.966472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:08.966343    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 16:33:09.021231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.020868    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:33:09.071754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.068165    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:09.163930 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.163903    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:33:09.368341 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.368298    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:09.369956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.369923    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:09.373749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.370397    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 03 16:33:09.377142 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.377123    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:33:09.419867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.419823    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:09.522749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.514923    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:09.551393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.547855    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 03 16:33:09.556775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.556710    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:33:09.556948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.556793    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:09.557353 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.557320    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:33:09.577825 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.577763    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:33:09.646335 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.635206    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 16:33:09.646335 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.635264    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 16:33:09.666243 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.666206    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:33:09.666934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.666912    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:33:09.673745 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.672975    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:33:09.680040 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.680019    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:33:09.729817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.710339    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:33:09.959088 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.958996    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 16:33:09.959304 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:09.959272    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 16:33:10.626192 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:10.626152    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 16:33:10.626798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:10.626762    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 16:33:10.958935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:10.958846    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 16:33:10.958935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:10.958905    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 16:33:11.621527 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:11.621490    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 16:33:11.621675 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:11.621546    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 16:33:11.959124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:11.959031    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 16:33:11.959124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:11.959093    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 16:33:13.663878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:13.663838    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:13.664350 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:13.663891    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:14.005041 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:14.004935    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:14.005338 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:14.005296    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:14.231068 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:14.231026    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:14.231987 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:14.231961    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:14.232744 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:14.232707    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:33:14.311652 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:14.311580    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:14.312103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:14.312075    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:14.312877 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:14.312851    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:33:25.011882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.011843    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"]
Nov 03 16:33:25.017074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.017047    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.064898 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.064866    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"]
Nov 03 16:33:25.069581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.069550    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.114941 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.114898    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"]
Nov 03 16:33:25.151023 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.150991    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"]
Nov 03 16:33:25.172689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.172656    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.172689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.172691    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.173028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.172714    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.173028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.172747    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.173028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.172778    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.173028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.172942    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab094755-d4d2-40bc-a411-e73e6fd3372f-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.173028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.172970    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c153a4f-9b14-4934-84a8-ba29826e20dd-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.173028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.172995    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.173367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.173080    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.173367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.173126    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.173367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.173145    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kq5f\" (UniqueName: \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-kube-api-access-2kq5f\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.173367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.173260    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.173367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.173302    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.173367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.173319    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.173367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.173335    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.173773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.173413    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mklg\" (UniqueName: \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-kube-api-access-7mklg\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.173773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.173475    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.173773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.173519    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.274451 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274366    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.274451 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274408    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.274451 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274427    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.274451 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274447    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.274819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274466    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.274819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274493    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.274819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274560    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab094755-d4d2-40bc-a411-e73e6fd3372f-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.274819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274591    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c153a4f-9b14-4934-84a8-ba29826e20dd-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.274819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274617    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.274819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274646    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.274819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274682    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.274819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274698    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kq5f\" (UniqueName: \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-kube-api-access-2kq5f\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.274819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274769    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.274819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274810    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.275329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274839    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.275329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274850    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.275329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274869    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.275329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274866    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.275329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.274961    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mklg\" (UniqueName: \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-kube-api-access-7mklg\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.275329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.275020    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-credential-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.275329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.275051    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.275329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.275322    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.275754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.275438    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-socket\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.275754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.275695    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-certs\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.275858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.275770    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.275894 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.275848    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-data\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.275894 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.275862    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab094755-d4d2-40bc-a411-e73e6fd3372f-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.275959 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.275915    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c153a4f-9b14-4934-84a8-ba29826e20dd-istiod-ca-cert\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.277295 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.277273    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.277627 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.277604    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.277904 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.277886    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-envoy\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.278149 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.278132    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-podinfo\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.335947 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.335915    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.336065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.335959    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mklg\" (UniqueName: \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-kube-api-access-7mklg\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.336065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.335919    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-token\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.336065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.336030    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kq5f\" (UniqueName: \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-kube-api-access-2kq5f\") pod \"e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") " pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.385943 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.385921    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:25.631775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.631718    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:25.823530 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.823501    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"]
Nov 03 16:33:25.824617 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:33:25.824593    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c153a4f_9b14_4934_84a8_ba29826e20dd.slice/crio-43ff7a8d12ba83ae393eaec3e2d9e8e1d56003d63a5ae202a3d3aed8cb9c81d6 WatchSource:0}: Error finding container 43ff7a8d12ba83ae393eaec3e2d9e8e1d56003d63a5ae202a3d3aed8cb9c81d6: Status 404 returned error can't find the container with id 43ff7a8d12ba83ae393eaec3e2d9e8e1d56003d63a5ae202a3d3aed8cb9c81d6
Nov 03 16:33:25.827104 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.827067    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:25.827188 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.827134    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:25.827188 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:25.827166    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:26.018794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:26.018771    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"]
Nov 03 16:33:26.021339 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:33:26.021303    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab094755_d4d2_40bc_a411_e73e6fd3372f.slice/crio-50ad0bf23ef962e0100a3b2c68e465483135bbd888e6e0f9ee39dfa478137eb6 WatchSource:0}: Error finding container 50ad0bf23ef962e0100a3b2c68e465483135bbd888e6e0f9ee39dfa478137eb6: Status 404 returned error can't find the container with id 50ad0bf23ef962e0100a3b2c68e465483135bbd888e6e0f9ee39dfa478137eb6
Nov 03 16:33:26.023813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:26.023782    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:26.023897 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:26.023848    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:26.023934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:26.023898    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:26.735006 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:26.734970    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" event={"ID":"ab094755-d4d2-40bc-a411-e73e6fd3372f","Type":"ContainerStarted","Data":"7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca"}
Nov 03 16:33:26.735212 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:26.735031    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" event={"ID":"ab094755-d4d2-40bc-a411-e73e6fd3372f","Type":"ContainerStarted","Data":"50ad0bf23ef962e0100a3b2c68e465483135bbd888e6e0f9ee39dfa478137eb6"}
Nov 03 16:33:26.736900 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:26.736872    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" event={"ID":"5c153a4f-9b14-4934-84a8-ba29826e20dd","Type":"ContainerStarted","Data":"8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc"}
Nov 03 16:33:26.737030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:26.736919    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" event={"ID":"5c153a4f-9b14-4934-84a8-ba29826e20dd","Type":"ContainerStarted","Data":"43ff7a8d12ba83ae393eaec3e2d9e8e1d56003d63a5ae202a3d3aed8cb9c81d6"}
Nov 03 16:33:26.863365 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:26.863314    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" podStartSLOduration=2.863293925 podStartE2EDuration="2.863293925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:33:26.85991895 +0000 UTC m=+1292.684370220" watchObservedRunningTime="2026-11-03 16:33:26.863293925 +0000 UTC m=+1292.687745195"
Nov 03 16:33:27.014793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:27.014641    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" podStartSLOduration=3.014617319 podStartE2EDuration="3.014617319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:33:26.990084496 +0000 UTC m=+1292.814535787" watchObservedRunningTime="2026-11-03 16:33:27.014617319 +0000 UTC m=+1292.839068592"
Nov 03 16:33:27.401752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:27.400825    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:27.402280 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:27.402167    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 16:33:27.402280 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:27.402235    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 16:33:27.635196 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:27.635111    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:27.638307 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:27.638158    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 16:33:27.638307 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:27.638204    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 16:33:27.922699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:27.922664    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"]
Nov 03 16:33:27.934210 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:27.934183    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.057031 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.056996    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"]
Nov 03 16:33:28.128753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.091257    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-podinfo\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.128753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.091333    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-token\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.128753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.091365    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-data\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.128753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.091420    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srtv2\" (UniqueName: \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-kube-api-access-srtv2\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.128753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.091689    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-credential-socket\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.128753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.091713    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-certs\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.128753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.091772    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-socket\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.128753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.091825    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afeb5a90-adb0-4757-828a-1cdf537c1414-istiod-ca-cert\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.128753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.091850    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-envoy\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.187798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.186338    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e998bdb4-openshift-default-dockercfg-26t82\""
Nov 03 16:33:28.195711 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.195650    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-credential-socket\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.195965 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.195943    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-certs\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.196143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.196102    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-socket\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.196340 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.196310    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afeb5a90-adb0-4757-828a-1cdf537c1414-istiod-ca-cert\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.196475 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.196457    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-envoy\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.196629 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.196610    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-podinfo\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.197010 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.196891    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-token\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.197010 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.196943    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-data\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.227533 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.197260    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srtv2\" (UniqueName: \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-kube-api-access-srtv2\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.227533 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.204479    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afeb5a90-adb0-4757-828a-1cdf537c1414-istiod-ca-cert\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.227533 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.204786    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-credential-socket\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.227533 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.205039    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-certs\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.227533 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.205264    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-socket\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.227533 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.206719    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-data\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.352163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.352129    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-token\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.395101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.388577    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 16:33:28.395101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.388642    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 16:33:28.504935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.494263    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-envoy\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.504935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.494965    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srtv2\" (UniqueName: \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-kube-api-access-srtv2\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.533197 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.523524    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-podinfo\") pod \"e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") " pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:28.649929 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.635741    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 16:33:28.651219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.650860    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 16:33:28.655577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:28.654592    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:29.158379 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:29.158317    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"]
Nov 03 16:33:29.162123 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:33:29.162095    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafeb5a90_adb0_4757_828a_1cdf537c1414.slice/crio-8dc06b51338ee9e5df5283f3c8de43268d79daea55645db160ceff80d7222694 WatchSource:0}: Error finding container 8dc06b51338ee9e5df5283f3c8de43268d79daea55645db160ceff80d7222694: Status 404 returned error can't find the container with id 8dc06b51338ee9e5df5283f3c8de43268d79daea55645db160ceff80d7222694
Nov 03 16:33:29.164846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:29.164797    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:29.164933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:29.164884    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:29.164978 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:29.164929    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:29.399749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:29.399048    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 16:33:29.399749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:29.399117    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 16:33:29.643756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:29.640174    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 16:33:29.643756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:29.640235    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 16:33:29.869295 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:29.869259    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" event={"ID":"afeb5a90-adb0-4757-828a-1cdf537c1414","Type":"ContainerStarted","Data":"38077d001c7df2dd48d513f8988962f88b58fd1a7125bda0fb118f6099da05c1"}
Nov 03 16:33:29.869527 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:29.869318    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" event={"ID":"afeb5a90-adb0-4757-828a-1cdf537c1414","Type":"ContainerStarted","Data":"8dc06b51338ee9e5df5283f3c8de43268d79daea55645db160ceff80d7222694"}
Nov 03 16:33:30.006807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:30.006598    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" podStartSLOduration=3.006577094 podStartE2EDuration="3.006577094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:33:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:33:30.003044173 +0000 UTC m=+1295.827495445" watchObservedRunningTime="2026-11-03 16:33:30.006577094 +0000 UTC m=+1295.831028367"
Nov 03 16:33:30.389895 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:30.389860    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 16:33:30.390210 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:30.390180    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 16:33:30.632678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:30.632640    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 16:33:30.633010 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:30.632978    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 16:33:30.655571 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:30.655500    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:30.661617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:30.661596    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:30.880217 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:30.880183    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:30.897783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:30.896036    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:33:31.406753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.401686    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 16:33:31.406753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.401779    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 16:33:31.633080 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.632704    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 16:33:31.633080 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.632787    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 16:33:31.873921 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.873885    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"]
Nov 03 16:33:31.877944 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.877923    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:31.897992 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.897962    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"]
Nov 03 16:33:31.904261 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.904236    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:31.957030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.957004    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"]
Nov 03 16:33:31.960792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.960763    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"]
Nov 03 16:33:31.978664 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.978633    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:31.978919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.978895    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:31.979120 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.979101    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e21a973c-98db-465f-b02e-f8e961c420ec-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:31.979287 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.979270    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:31.979440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.979419    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:31.979531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.979449    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4mqd\" (UniqueName: \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-kube-api-access-z4mqd\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:31.979531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.979496    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:31.979643 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.979533    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:31.979643 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:31.979571    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e21a973c-98db-465f-b02e-f8e961c420ec-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.080401 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.080365    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.080401 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.080404    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.080641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.080447    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.080641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.080468    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.080641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.080492    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.080641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.080522    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e21a973c-98db-465f-b02e-f8e961c420ec-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.081007    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.082924    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.082983    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.083017    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwwh8\" (UniqueName: \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-kube-api-access-cwwh8\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.083096    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.083404    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.083441    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.083491    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.083536    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e21a973c-98db-465f-b02e-f8e961c420ec-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.083564    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.083647    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.083707    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.083760    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4mqd\" (UniqueName: \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-kube-api-access-z4mqd\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.084986    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.096755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.085250    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.120751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.098774    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e21a973c-98db-465f-b02e-f8e961c420ec-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.120751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.099154    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.138601 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.138484    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e21a973c-98db-465f-b02e-f8e961c420ec-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.138784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.138614    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.148369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.148333    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.150703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.150676    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4mqd\" (UniqueName: \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-kube-api-access-z4mqd\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.195752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.193797    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.195752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.193865    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwwh8\" (UniqueName: \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-kube-api-access-cwwh8\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.195752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.193953    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.195752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.193996    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.195752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.194032    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.195752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.194088    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.195752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.194316    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.195752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.194483    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.199751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.197952    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.223749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.219139    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istiod-ca-cert\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.223749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.219486    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-data\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.223749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.220229    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-certs\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.223749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.220275    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-credential-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.223749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.220423    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-socket\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.236752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.233235    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:32.243209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.239081    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-envoy\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.244175 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.244112    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-podinfo\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.332755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.314484    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwwh8\" (UniqueName: \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-kube-api-access-cwwh8\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.341755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.338501    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-token\") pod \"e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") " pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.418813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.417763    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 16:33:32.418813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.417822    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 16:33:32.561750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.553913    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:32.640753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.637178    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 16:33:32.640753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.637240    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 16:33:32.857162 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.855207    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"]
Nov 03 16:33:32.867443 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.867109    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:32.867443 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.867198    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:32.867443 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.867238    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:32.911436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:32.911400    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" event={"ID":"e21a973c-98db-465f-b02e-f8e961c420ec","Type":"ContainerStarted","Data":"ede00f6d9556ca4fcaaea2fc8ae1cecba1005b5677e95a8f0bd5151440a06ad0"}
Nov 03 16:33:33.210030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.193811    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"]
Nov 03 16:33:33.210030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.197705    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"]
Nov 03 16:33:33.210030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.199595    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:33.210030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.199702    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:33.210030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.199823    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:33.210030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.203890    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.287436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.287409    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1b774106-openshift-default-dockercfg-6kvxx\""
Nov 03 16:33:33.294043 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.294016    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"]
Nov 03 16:33:33.340319 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.340283    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-data\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.340477 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.340409    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-envoy\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.340570 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.340539    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-socket\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.340898 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.340865    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-credential-socket\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.341011 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.340930    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-podinfo\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.341011 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.340965    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88ftj\" (UniqueName: \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-kube-api-access-88ftj\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.341138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.341056    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-token\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.341199 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.341140    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-certs\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.341259 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.341218    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/374d3275-56a4-4abf-89e3-34e7ee57294c-istiod-ca-cert\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.388744 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.388275    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 03 16:33:33.388744 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.388332    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 03 16:33:33.411482 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:33:33.411403    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:33:33.445170 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.445070    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-credential-socket\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.474752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.468961    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-credential-socket\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.474752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.472139    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-podinfo\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.474752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.472898    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88ftj\" (UniqueName: \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-kube-api-access-88ftj\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.474752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.472961    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-token\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.488796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.487021    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-certs\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.488796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.487103    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/374d3275-56a4-4abf-89e3-34e7ee57294c-istiod-ca-cert\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.488796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.487228    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-data\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.488796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.487329    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-envoy\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.488796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.487473    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-socket\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.488796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.488663    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-socket\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.489259 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.489032    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-data\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.489259 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.489083    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/374d3275-56a4-4abf-89e3-34e7ee57294c-istiod-ca-cert\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.499160 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.499080    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-certs\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.514752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.510932    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-envoy\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.531758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.529672    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-podinfo\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.550756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.541390    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88ftj\" (UniqueName: \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-kube-api-access-88ftj\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.558750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.551302    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-token\") pod \"e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") " pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.635223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.632571    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 16:33:33.635223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.632634    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 16:33:33.844756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.841075    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:33.949074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.949019    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" event={"ID":"e21a973c-98db-465f-b02e-f8e961c420ec","Type":"ContainerStarted","Data":"a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966"}
Nov 03 16:33:33.951770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.951701    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" event={"ID":"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8","Type":"ContainerStarted","Data":"b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80"}
Nov 03 16:33:33.951909 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:33.951777    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" event={"ID":"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8","Type":"ContainerStarted","Data":"986dee5bb2babc20c85ad741ed4efc28249879ed4b758b57acfc045dfb3558ba"}
Nov 03 16:33:34.254807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.254070    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podStartSLOduration=3.254046258 podStartE2EDuration="3.254046258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:33:34.109191432 +0000 UTC m=+1299.933642703" watchObservedRunningTime="2026-11-03 16:33:34.254046258 +0000 UTC m=+1300.078497529"
Nov 03 16:33:34.265857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.258950    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:34.265857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.260862    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 16:33:34.265857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.260908    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 16:33:34.265857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.261399    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podStartSLOduration=3.261383942 podStartE2EDuration="3.261383942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:33:34.236218796 +0000 UTC m=+1300.060670108" watchObservedRunningTime="2026-11-03 16:33:34.261383942 +0000 UTC m=+1300.085835213"
Nov 03 16:33:34.440386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.435019    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"]
Nov 03 16:33:34.460943 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.458394    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:34.460943 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.458523    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:34.460943 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.458594    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:33:34.535810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.516501    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:34.537852 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.537779    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:34.580768 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.570042    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:33:34.582653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.582625    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:34.594538 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.594457    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 16:33:34.594538 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.594514    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 16:33:34.910787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.905967    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:34.914757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.914711    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:34.921955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:34.917557    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:33:35.152896 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:35.152808    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" event={"ID":"374d3275-56a4-4abf-89e3-34e7ee57294c","Type":"ContainerStarted","Data":"9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b"}
Nov 03 16:33:35.155193 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:35.155169    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" event={"ID":"374d3275-56a4-4abf-89e3-34e7ee57294c","Type":"ContainerStarted","Data":"4b107fdca29f338d53cb304ba6ac0fa2255be15bfc659378dffd455783237574"}
Nov 03 16:33:35.244876 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:35.244839    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 16:33:35.245114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:35.245084    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 16:33:35.561819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:35.561118    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 16:33:35.561819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:35.561197    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 16:33:35.842350 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:35.842317    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:35.847698 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:35.847669    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:36.089751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:36.086907    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" podStartSLOduration=3.086883199 podStartE2EDuration="3.086883199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:33:35.593363331 +0000 UTC m=+1301.417814603" watchObservedRunningTime="2026-11-03 16:33:36.086883199 +0000 UTC m=+1301.911334473"
Nov 03 16:33:36.219473 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:36.214613    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:36.230305 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:36.230275    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:33:36.251923 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:36.251888    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 16:33:36.252174 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:36.252144    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 16:33:36.739753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:36.678982    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 16:33:36.739753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:36.679159    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 16:33:37.239707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:37.239675    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 16:33:37.240108 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:37.239743    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 16:33:37.568887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:37.561864    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 16:33:37.568887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:37.561926    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 16:33:38.239886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:38.239845    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 16:33:38.240340 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:38.239901    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 16:33:38.559845 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:38.558256    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 16:33:38.559845 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:38.558320    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 16:33:39.240049 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:39.240013    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 16:33:39.240460 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:39.240063    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 16:33:39.555098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:39.555008    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 16:33:39.555098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:39.555073    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 16:33:40.247379 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:40.247327    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Nov 03 16:33:40.247968 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:40.247401    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Nov 03 16:33:40.555006 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:40.554907    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 16:33:40.555006 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:40.554984    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 16:33:41.563759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:41.559837    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 16:33:41.563759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:41.559899    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 16:33:42.167620 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:42.165834    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:42.292755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:42.287826    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:42.292755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:42.287892    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:42.388533 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:42.388490    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:42.389353 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:42.388942    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:42.464930 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:42.464825    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:42.498152 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:42.498103    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:42.498610 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:42.498403    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:42.558920 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:42.558875    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:42.750676 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:42.750566    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:43.201837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.201774    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/health\": context deadline exceeded"
Nov 03 16:33:43.202517 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.202470    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 16:33:43.203134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.203091    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": context deadline exceeded"
Nov 03 16:33:43.209643 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.209602    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:43.220757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.217210    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:33:43.220757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.219701    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 03 16:33:43.246756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.245920    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/health\": context deadline exceeded"
Nov 03 16:33:43.476810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.475695    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:43.476810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.476578    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:33:43.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.526400    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 16:33:43.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.526880    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:43.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.527215    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:33:43.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.527269    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:33:43.529803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.529521    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:33:43.619781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.531851    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:33:43.619781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.533465    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:43.619781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.533789    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:33:43.619781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.559981    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:43.619781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.578558    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:33:43.619781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.606875    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:43.619781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.606936    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:43.643092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.642108    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:43.824017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.823932    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:43.824606 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:43.824576    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:44.240754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.223897    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:44.240754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.226784    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 16:33:44.249750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.245468    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:33:44.292801 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.275686    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 03 16:33:44.292801 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.289379    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:33:44.557937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.554899    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 03 16:33:44.558218 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.558184    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:44.649753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.562238    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 03 16:33:44.649753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.562267    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 16:33:44.649753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.562300    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 03 16:33:44.649753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.563473    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:44.649753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.648867    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:44.835822 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.829925    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 16:33:44.917975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.917935    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:44.918249 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.918219    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:44.930767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.927175    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:44.996763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:44.986539    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:45.260051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.247861    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 16:33:45.260051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.248044    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 16:33:45.288873 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.288824    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 03 16:33:45.515935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.515845    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:45.516274 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.516238    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:45.609054 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.609004    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:45.615755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.611543    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:45.656956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.641905    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:33:45.656956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.642041    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:45.656956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.642107    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 16:33:45.656956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.645872    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:45.658078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.658040    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:45.742756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.741965    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:45.742756 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:33:45.742343    5605 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.432s"
Nov 03 16:33:45.742756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.742527    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:33:45.831230 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.831128    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:45.842701 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.833652    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:33:45.948945 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.948902    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:45.949158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:45.948976    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:46.034879 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.034838    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:46.035139 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.035112    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:46.035714 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.035685    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/health\": context deadline exceeded"
Nov 03 16:33:46.685108 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.685063    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:46.685746 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.685690    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:46.726927 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.726885    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:46.727542 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.727509    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 03 16:33:46.727827 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.727801    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:46.728210 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.728188    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:46.728596 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.728565    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:46.728995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.728970    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 03 16:33:46.734849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.734815    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 16:33:46.760021 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.757828    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 03 16:33:46.848758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.831822    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 16:33:46.980978 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.980883    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:46.981276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:46.981225    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:47.071453 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.071414    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:47.071713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.071687    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:47.338819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.318772    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:47.386750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.383492    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:47.554867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.554834    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:47.639570 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.639447    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:47.695756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.685454    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:47.695756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.686258    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:47.756756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.730865    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:47.756756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.731272    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 03 16:33:47.756756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.731554    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:47.756756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.731816    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 16:33:47.756756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.732054    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 03 16:33:47.756756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.732287    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:33:47.759707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.758153    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:47.759707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.758988    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:47.836789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:47.832020    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:48.029369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.028878    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:48.029369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.028942    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:48.077292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.076844    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:48.077292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.076909    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:48.656176 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.656135    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:48.692098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.692057    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:48.692694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.692663    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 16:33:48.766128 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.766079    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:48.825034 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.766619    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 03 16:33:48.846075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.768644    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 03 16:33:48.846371 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.768684    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:48.846679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.768707    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:48.847223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.768752    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 16:33:48.847708 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.768774    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 03 16:33:48.854805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.768791    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:33:48.863761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:48.845947    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:49.318756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.297995    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:49.318756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.298098    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:49.318756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.317946    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:49.318756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.318056    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:49.730813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.478420    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:49.730813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.478925    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/health\": context deadline exceeded"
Nov 03 16:33:49.744059 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.744002    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:49.832289 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.832244    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:50.066942 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.850848    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:50.071837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.850935    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:50.078297 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.850968    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 03 16:33:50.086830 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.851125    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:50.106819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.890927    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:33:50.110665 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:49.890996    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 16:33:50.112292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.112239    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:50.113607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.113553    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:50.119012 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.118966    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 03 16:33:50.456418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.453964    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:50.456418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.454027    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:50.551753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.548019    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:50.552095 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.552036    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:50.576906 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.553038    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:33:50.632297 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.632254    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:33:50.703935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.703899    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:50.710914 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.710816    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:50.747393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:50.747353    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:51.026068 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.025971    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:51.076318 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.076254    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:51.085973 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.074402    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 03 16:33:51.086408 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.073360    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:51.086690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.086212    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 03 16:33:51.113098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.112546    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:51.113098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.112860    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:51.113472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.112458    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:51.114076 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.113873    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 16:33:51.166771 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.138741    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:51.305800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.300077    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:51.411160 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.411125    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:33:51.500991 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.497791    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:33:51.500991 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.497854    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:51.638343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.638313    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:33:51.647029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.640810    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:33:51.701639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.701595    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:33:51.730250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.730218    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:33:51.737557 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.737531    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:33:51.741444 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.741412    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:33:51.750080 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.750058    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:33:51.753816 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.753795    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:33:51.774796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.755867    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:33:51.774796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.760134    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:33:51.833674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.833636    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:33:51.835899 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:33:51.835873    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:34:03.393434 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:34:03.393386    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:34:14.749752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:14.719348    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"]
Nov 03 16:34:14.879124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:14.879086    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"]
Nov 03 16:34:14.879493 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:14.879470    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:14.999872 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:14.991099    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e998bdb4-serving-cert\""
Nov 03 16:34:15.078752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.078445    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-maas-api-tls\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.078752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.078574    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-metrics-certs\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.078752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.078651    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9196ae2-3beb-4687-a400-98a612a4988d-openshift-service-ca\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.078752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.078675    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-997dk\" (UniqueName: \"kubernetes.io/projected/b9196ae2-3beb-4687-a400-98a612a4988d-kube-api-access-997dk\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.197306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.196148    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-maas-api-tls\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.197306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.196283    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-metrics-certs\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.197306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.196493    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9196ae2-3beb-4687-a400-98a612a4988d-openshift-service-ca\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.197306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.196526    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997dk\" (UniqueName: \"kubernetes.io/projected/b9196ae2-3beb-4687-a400-98a612a4988d-kube-api-access-997dk\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.235752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.220559    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9196ae2-3beb-4687-a400-98a612a4988d-openshift-service-ca\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.240201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.240171    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-maas-api-tls\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.262892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.262803    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-metrics-certs\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.299263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.299228    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-997dk\" (UniqueName: \"kubernetes.io/projected/b9196ae2-3beb-4687-a400-98a612a4988d-kube-api-access-997dk\") pod \"maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.556603 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.556505    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 03 16:34:15.577254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.577216    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:15.709767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.705227    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 16:34:15.790483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.790445    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"]
Nov 03 16:34:15.831684 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.831605    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:15.942243 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.942173    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-envoy\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:15.942565 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.942527    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43263885-de5b-4337-848e-39e6bd270649-istiod-ca-cert\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:15.942909 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.942891    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-istio-token\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:15.943090 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.943073    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-credential-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:15.943268 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.943253    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:15.943370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.943358    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcvg5\" (UniqueName: \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-kube-api-access-wcvg5\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:15.943522 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.943501    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43263885-de5b-4337-848e-39e6bd270649-istio-podinfo\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:15.943681 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.943665    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-certs\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:15.943925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:15.943908    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-data\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.066901 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.066864    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-istio-token\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.067250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.067226    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-credential-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.067468 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.067451    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.067593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.067576    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcvg5\" (UniqueName: \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-kube-api-access-wcvg5\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.067800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.067781    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43263885-de5b-4337-848e-39e6bd270649-istio-podinfo\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.067938 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.067923    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-certs\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.073761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.068125    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-data\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.074030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.074002    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-envoy\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.078747 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.074185    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43263885-de5b-4337-848e-39e6bd270649-istiod-ca-cert\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.123372 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.123339    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.124744 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.124611    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43263885-de5b-4337-848e-39e6bd270649-istiod-ca-cert\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.213753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.161165    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-certs\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.213753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.162117    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-data\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.253501 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.214426    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"]
Nov 03 16:34:16.253501 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.249123    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-credential-socket\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.282903 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.282872    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcvg5\" (UniqueName: \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-kube-api-access-wcvg5\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.347748 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.347671    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43263885-de5b-4337-848e-39e6bd270649-istio-podinfo\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.378856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.370131    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-envoy\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.450751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.450128    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-istio-token\") pod \"e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") " pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.475938 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.475899    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"]
Nov 03 16:34:16.559784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.543396    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:16.576786 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.560198    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:16.576786 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.561053    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:34:16.608858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.582647    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"]
Nov 03 16:34:16.633520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.633144    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:16.707083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.707019    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 16:34:16.710041 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.708495    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:16.710358 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.710339    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:16.710614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.710594    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:16.714849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.714809    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:16.715088 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.715068    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:16.717805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.709228    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:16.718285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.708820    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 03 16:34:16.718818 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.708984    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 16:34:16.719043 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.719021    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl8r5\" (UniqueName: \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-kube-api-access-bl8r5\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:16.719255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.719235    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:16.719444 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.719425    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:16.719581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.719565    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:16.777475 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.769771    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:34:16.777475 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.771144    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 03 16:34:16.777475 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.771228    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 03 16:34:16.911995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:16.904067    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 16:34:17.061292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.034833    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:34:17.061292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.049640    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"]
Nov 03 16:34:17.337811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.296567    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.339819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.339781    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.340309 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.340285    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.364753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.340466    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.365404 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.365093    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.365698 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.365679    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl8r5\" (UniqueName: \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-kube-api-access-bl8r5\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.369751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.366123    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.385760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.370292    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.427756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.427489    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.462897 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.462845    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 16:34:17.512026 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.511423    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-certs\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.547917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.511769    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-data\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.574112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.574061    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:17.811805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.805455    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.811805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.805803    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-credential-socket\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:17.947913 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:17.947855    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:18.273285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.273244    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istiod-ca-cert\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:18.273534 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.137674    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:18.274050 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.139782    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-podinfo\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:18.274221 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.142375    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:18.274346 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.142401    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:18.296749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.166977    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.044837    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.168066    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.170000    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.170019    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.208432    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.223315    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.242251    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"]
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.313178    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.313221    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.313254    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.313289    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.313321    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:34:18.353751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.313352    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:34:18.410600 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.169543    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:18.588138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.588105    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 03 16:34:18.606498 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.591929    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 16:34:18.712653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.711949    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl8r5\" (UniqueName: \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-kube-api-access-bl8r5\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:18.713264 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.713231    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-envoy\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:18.939023 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.714426    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:34:18.939347 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.939326    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"]
Nov 03 16:34:18.939624 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.939592    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:34:18.939864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.716220    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-podinfo\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:18.940270 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.940245    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istiod-ca-cert\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:18.940570 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.940530    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:18.941762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.940777    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-certs\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:18.942257 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.942233    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-credential-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:18.945947 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.945924    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-envoy\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:18.946485 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.946443    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-token\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:18.978962 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.978873    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-data\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.010794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:18.989120    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxqvn\" (UniqueName: \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-kube-api-access-mxqvn\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.090423    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-credential-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.090494    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-envoy\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.090614    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-token\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.090662    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-data\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.090775    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxqvn\" (UniqueName: \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-kube-api-access-mxqvn\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.090835    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-podinfo\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.090942    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istiod-ca-cert\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.090996    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.091035    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-certs\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.091589    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-certs\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.093697    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.117846    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istiod-ca-cert\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.127142    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-data\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.135823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.135031    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-credential-socket\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.137254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.137190    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-envoy\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.181764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.162199    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/health\": context deadline exceeded"
Nov 03 16:34:19.207106 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.184358    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:34:19.207106 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.184868    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-token\") pod \"e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") " pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:19.247629 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.247550    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-podinfo\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.273487 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.248428    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:34:19.365764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.335762    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld"
Nov 03 16:34:19.365764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.263244    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 16:34:19.365764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.267617    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-token\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.365764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.337520    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:34:19.365764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.343661    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxqvn\" (UniqueName: \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-kube-api-access-mxqvn\") pod \"e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") " pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.365764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.343760    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:19.366379 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.366342    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 16:34:19.382815 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.368610    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:19.478109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.368992    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:19.478109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.473395    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 16:34:19.478109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.404122    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 16:34:19.478109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.426337    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:19.478109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.442113    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:19.480069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.369829    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 03 16:34:19.480189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.480080    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:34:19.580553 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.580427    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:34:19.580553 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.580491    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:34:19.583606 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.583566    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 16:34:19.625496 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.625463    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:34:19.628680 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.628656    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:34:19.649513 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.640666    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:34:19.649513 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.644440    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t" event={"ID":"b9196ae2-3beb-4687-a400-98a612a4988d","Type":"ContainerStarted","Data":"dfa4df8051a1d3a142f9fbc088a83bb20ab2590c69a8d3e3bed2d69b00c042dc"}
Nov 03 16:34:19.822167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.821607    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"]
Nov 03 16:34:19.835201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.826372    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:34:19.835201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.826462    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:34:19.835201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.826498    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:34:19.967151 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:34:19.966368    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb423ba_e1b2_4f9d_bae0_8b189990ec26.slice/crio-6881a61e1feb6bffc74fd48f7d3fafdec219bedbf15f1af5d4b53cc25d2d7bc8 WatchSource:0}: Error finding container 6881a61e1feb6bffc74fd48f7d3fafdec219bedbf15f1af5d4b53cc25d2d7bc8: Status 404 returned error can't find the container with id 6881a61e1feb6bffc74fd48f7d3fafdec219bedbf15f1af5d4b53cc25d2d7bc8
Nov 03 16:34:19.970072 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.969575    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:34:19.970072 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.969646    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:34:19.970072 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.969673    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:34:19.973880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:19.973463    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"]
Nov 03 16:34:20.052830 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.052802    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"]
Nov 03 16:34:20.054101 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:34:20.054078    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9c2e2d_9e39_434c_b2b5_db3dd5cb0019.slice/crio-7bd0c1295679b81acb98a00758d4549afe163510b7c364ba5a74596b0933afb8 WatchSource:0}: Error finding container 7bd0c1295679b81acb98a00758d4549afe163510b7c364ba5a74596b0933afb8: Status 404 returned error can't find the container with id 7bd0c1295679b81acb98a00758d4549afe163510b7c364ba5a74596b0933afb8
Nov 03 16:34:20.056846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.056801    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:34:20.056975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.056880    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:34:20.056975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.056927    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:34:20.665678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.665428    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t" event={"ID":"b9196ae2-3beb-4687-a400-98a612a4988d","Type":"ContainerStarted","Data":"bfb68dace4fc7a70dc3885bd0da9aa5ca5f3911f039c431df88fc314ccc5ba36"}
Nov 03 16:34:20.666208 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.666172    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:20.668246 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.668212    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" event={"ID":"43263885-de5b-4337-848e-39e6bd270649","Type":"ContainerStarted","Data":"1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9"}
Nov 03 16:34:20.668354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.668263    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" event={"ID":"43263885-de5b-4337-848e-39e6bd270649","Type":"ContainerStarted","Data":"8b967cb1441728ce9709c9d5cd995ef151c71cc2bf96c989603b5bcde6e1cef0"}
Nov 03 16:34:20.669967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.669937    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" event={"ID":"4eb423ba-e1b2-4f9d-bae0-8b189990ec26","Type":"ContainerStarted","Data":"d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474"}
Nov 03 16:34:20.670079 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.669987    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" event={"ID":"4eb423ba-e1b2-4f9d-bae0-8b189990ec26","Type":"ContainerStarted","Data":"6881a61e1feb6bffc74fd48f7d3fafdec219bedbf15f1af5d4b53cc25d2d7bc8"}
Nov 03 16:34:20.672583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.672555    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" event={"ID":"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019","Type":"ContainerStarted","Data":"bd57e88bbcb3fa578ac45961573ae740901070347ca9861f4730f0fcfe6ac73c"}
Nov 03 16:34:20.672698 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:20.672602    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" event={"ID":"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019","Type":"ContainerStarted","Data":"7bd0c1295679b81acb98a00758d4549afe163510b7c364ba5a74596b0933afb8"}
Nov 03 16:34:21.419753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:21.394409    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:21.461793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:21.461759    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:21.479907 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:21.403109    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 16:34:21.480172 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:21.480146    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 16:34:21.483654 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:21.464985    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 16:34:21.483882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:21.483854    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 16:34:21.661955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:21.641475    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:21.662307 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:21.662278    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 16:34:21.662480 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:21.662455    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 16:34:22.067776 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:22.060705    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" podStartSLOduration=6.060683114 podStartE2EDuration="6.060683114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:34:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:34:22.059162687 +0000 UTC m=+1347.883613960" watchObservedRunningTime="2026-11-03 16:34:22.060683114 +0000 UTC m=+1347.885134386"
Nov 03 16:34:22.376748 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:22.374493    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t" podStartSLOduration=7.198469395 podStartE2EDuration="8.374469542s" totalImagesPullingTime="1.176000147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:34:22.20629158 +0000 UTC m=+1348.030742851" watchObservedRunningTime="2026-11-03 16:34:22.374469542 +0000 UTC m=+1348.198920815"
Nov 03 16:34:22.381897 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:22.381856    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 16:34:22.382180 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:22.382145    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 16:34:22.478639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:22.470795    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 16:34:22.478639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:22.470882    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 16:34:22.563360 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:22.527854    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" podStartSLOduration=7.527835404 podStartE2EDuration="7.527835404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:34:22.353829119 +0000 UTC m=+1348.178280392" watchObservedRunningTime="2026-11-03 16:34:22.527835404 +0000 UTC m=+1348.352286676"
Nov 03 16:34:22.568827 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:22.568765    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" podStartSLOduration=6.568743871 podStartE2EDuration="6.568743871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:34:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:34:22.525212799 +0000 UTC m=+1348.349664100" watchObservedRunningTime="2026-11-03 16:34:22.568743871 +0000 UTC m=+1348.393195139"
Nov 03 16:34:22.635083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:22.634978    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 16:34:22.635355 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:22.635309    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 16:34:23.370865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:23.370830    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 16:34:23.371173 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:23.371138    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 16:34:23.444169 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:23.444124    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 16:34:23.444376 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:23.444190    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 16:34:23.643817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:23.643408    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 16:34:23.643817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:23.643470    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 16:34:24.424793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:24.422238    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 16:34:24.424793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:24.422304    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 16:34:24.460754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:24.443545    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 16:34:24.460754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:24.443618    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 16:34:24.659911 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:24.659874    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 16:34:24.660794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:24.660758    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 16:34:25.371469 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:25.371265    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 16:34:25.374665 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:25.373899    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 16:34:25.458184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:25.458144    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 16:34:25.458467 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:25.458436    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 16:34:25.634964 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:25.634856    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 16:34:25.634964 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:25.634938    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 16:34:26.448970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.448919    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 16:34:26.449574 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.449536    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 16:34:26.456761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.450078    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 16:34:26.456761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.450138    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 16:34:26.523547 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.523517    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"]
Nov 03 16:34:26.642757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.616614    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"]
Nov 03 16:34:26.728912 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.668257    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:34:26.728912 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.688649    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 16:34:26.728912 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.688702    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 16:34:26.769154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.764184    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"]
Nov 03 16:34:26.784820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.770157    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/700a9035-03d8-4cb2-a9e3-c0718f114297-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl\" (UID: \"700a9035-03d8-4cb2-a9e3-c0718f114297\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:34:26.797864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.797819    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k2zd\" (UniqueName: \"kubernetes.io/projected/700a9035-03d8-4cb2-a9e3-c0718f114297-kube-api-access-7k2zd\") pod \"payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl\" (UID: \"700a9035-03d8-4cb2-a9e3-c0718f114297\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:34:26.927752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.859173    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-161f6056\""
Nov 03 16:34:26.927752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.859330    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-161f6056-dockercfg-pmqd8\""
Nov 03 16:34:26.927752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.920197    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/700a9035-03d8-4cb2-a9e3-c0718f114297-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl\" (UID: \"700a9035-03d8-4cb2-a9e3-c0718f114297\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:34:26.927752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.920275    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k2zd\" (UniqueName: \"kubernetes.io/projected/700a9035-03d8-4cb2-a9e3-c0718f114297-kube-api-access-7k2zd\") pod \"payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl\" (UID: \"700a9035-03d8-4cb2-a9e3-c0718f114297\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:34:26.966755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.951645    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/700a9035-03d8-4cb2-a9e3-c0718f114297-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl\" (UID: \"700a9035-03d8-4cb2-a9e3-c0718f114297\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:34:26.995800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.981953    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"]
Nov 03 16:34:26.995800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:26.983110    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:34:27.125755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.112673    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k2zd\" (UniqueName: \"kubernetes.io/projected/700a9035-03d8-4cb2-a9e3-c0718f114297-kube-api-access-7k2zd\") pod \"payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl\" (UID: \"700a9035-03d8-4cb2-a9e3-c0718f114297\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:34:27.206773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.196035    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:34:27.206773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.205890    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:27.397711 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.397677    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 16:34:27.398065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.397982    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 16:34:27.423218 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.399601    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:27.482422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.476789    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09f0a14b-47aa-4887-855d-9b3b1600eb4d-plugins-config-volume\") pod \"payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq\" (UID: \"09f0a14b-47aa-4887-855d-9b3b1600eb4d\") " pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:34:27.482422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.477253    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q285d\" (UniqueName: \"kubernetes.io/projected/09f0a14b-47aa-4887-855d-9b3b1600eb4d-kube-api-access-q285d\") pod \"payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq\" (UID: \"09f0a14b-47aa-4887-855d-9b3b1600eb4d\") " pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:34:27.545752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.531820    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 16:34:27.561763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.558876    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Nov 03 16:34:27.568937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.568823    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 16:34:27.568937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.568880    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 16:34:27.571377 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.569236    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 03 16:34:27.571377 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.569233    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:27.583765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.579894    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q285d\" (UniqueName: \"kubernetes.io/projected/09f0a14b-47aa-4887-855d-9b3b1600eb4d-kube-api-access-q285d\") pod \"payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq\" (UID: \"09f0a14b-47aa-4887-855d-9b3b1600eb4d\") " pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:34:27.583765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.580068    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09f0a14b-47aa-4887-855d-9b3b1600eb4d-plugins-config-volume\") pod \"payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq\" (UID: \"09f0a14b-47aa-4887-855d-9b3b1600eb4d\") " pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:34:27.583765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.581297    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09f0a14b-47aa-4887-855d-9b3b1600eb4d-plugins-config-volume\") pod \"payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq\" (UID: \"09f0a14b-47aa-4887-855d-9b3b1600eb4d\") " pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:34:27.603766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.601625    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:27.642792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.642660    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:27.644008 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.643981    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 16:34:27.645668 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.644938    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 16:34:27.661789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.661753    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:34:27.689206 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.689110    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/health\": context deadline exceeded"
Nov 03 16:34:27.823426 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.773168    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q285d\" (UniqueName: \"kubernetes.io/projected/09f0a14b-47aa-4887-855d-9b3b1600eb4d-kube-api-access-q285d\") pod \"payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq\" (UID: \"09f0a14b-47aa-4887-855d-9b3b1600eb4d\") " pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:34:27.823426 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.773745    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"]
Nov 03 16:34:27.847752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.823949    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:34:27.964103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.964059    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:34:27.964406 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:27.964384    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.041791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.039390    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1b774106-serving-cert\""
Nov 03 16:34:28.102610 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.102560    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"]
Nov 03 16:34:28.183269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.182780    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f868573-448d-4ddf-8043-aa2c181800fd-openshift-service-ca\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.183269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.183000    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-maas-api-tls\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.183269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.183083    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcf8s\" (UniqueName: \"kubernetes.io/projected/6f868573-448d-4ddf-8043-aa2c181800fd-kube-api-access-gcf8s\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.183269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.183110    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-metrics-certs\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.381231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.356041    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-maas-api-tls\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.381231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.356214    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcf8s\" (UniqueName: \"kubernetes.io/projected/6f868573-448d-4ddf-8043-aa2c181800fd-kube-api-access-gcf8s\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.381231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.356264    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-metrics-certs\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.381231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.356326    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f868573-448d-4ddf-8043-aa2c181800fd-openshift-service-ca\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.381231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.357383    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f868573-448d-4ddf-8043-aa2c181800fd-openshift-service-ca\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.399751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.392482    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 03 16:34:28.419933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.419866    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 03 16:34:28.452069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.450861    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Nov 03 16:34:28.452069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.450919    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Nov 03 16:34:28.574350 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.574314    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-maas-api-tls\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.577306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.577279    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-metrics-certs\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.603591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.602634    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcf8s\" (UniqueName: \"kubernetes.io/projected/6f868573-448d-4ddf-8043-aa2c181800fd-kube-api-access-gcf8s\") pod \"maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:28.624344 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.618816    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"]
Nov 03 16:34:28.635708 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.635671    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 03 16:34:28.635863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.635760    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 03 16:34:28.672564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.672529    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"]
Nov 03 16:34:28.699189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:28.699149    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:29.124359 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:29.124328    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"]
Nov 03 16:34:29.546492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:29.546388    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq" event={"ID":"09f0a14b-47aa-4887-855d-9b3b1600eb4d","Type":"ContainerStarted","Data":"eb3ab9d5d6d77a54c01226616578330879392e0c8f20824f0966fa1e14d7c9ae"}
Nov 03 16:34:29.548630 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:29.548600    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq" event={"ID":"6f868573-448d-4ddf-8043-aa2c181800fd","Type":"ContainerStarted","Data":"512a995bd5bb696b17ad0fdf3a684a9f467a64e383530a48a35c42380d97a058"}
Nov 03 16:34:29.550247 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:29.550222    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl" event={"ID":"700a9035-03d8-4cb2-a9e3-c0718f114297","Type":"ContainerStarted","Data":"63f42599b00675df2dbf82c347691ab2f2f2782cba8c25f4d4e54eb36a3ddcf8"}
Nov 03 16:34:30.012769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.001086    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:30.012769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.009131    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:30.019891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.019821    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:34:30.044800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.042030    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:30.044800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.043649    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:30.265583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.262077    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:34:30.329208 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.328695    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:30.329460 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.329429    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:30.337862 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.337827    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:34:30.563923 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.563837    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq" event={"ID":"6f868573-448d-4ddf-8043-aa2c181800fd","Type":"ContainerStarted","Data":"33a83c3d10c5e3e1fa2de2437f2ddad48ea9e1e78e7f436c3b9ae727b23ccf73"}
Nov 03 16:34:30.564854 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.564827    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:30.856134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:30.856073    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq" podStartSLOduration=2.800927256 podStartE2EDuration="3.856053653s" totalImagesPullingTime="1.055126397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:34:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:34:30.855128873 +0000 UTC m=+1356.679580145" watchObservedRunningTime="2026-11-03 16:34:30.856053653 +0000 UTC m=+1356.680504948"
Nov 03 16:34:33.400628 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:34:33.400584    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:34:33.591979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:33.591941    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq" event={"ID":"09f0a14b-47aa-4887-855d-9b3b1600eb4d","Type":"ContainerStarted","Data":"00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716"}
Nov 03 16:34:33.592163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:33.592013    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:34:33.593693 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:33.593661    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl" event={"ID":"700a9035-03d8-4cb2-a9e3-c0718f114297","Type":"ContainerStarted","Data":"c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93"}
Nov 03 16:34:33.593837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:33.593720    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:34:33.718263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:33.714337    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq" podStartSLOduration=3.535299957 podStartE2EDuration="7.714316818s" totalImagesPullingTime="4.179016861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:34:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:34:33.712090464 +0000 UTC m=+1359.536541732" watchObservedRunningTime="2026-11-03 16:34:33.714316818 +0000 UTC m=+1359.538768091"
Nov 03 16:34:33.880969 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:33.877602    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl" podStartSLOduration=3.715134915 podStartE2EDuration="7.877581637s" totalImagesPullingTime="4.162446722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:34:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:34:33.875224236 +0000 UTC m=+1359.699675532" watchObservedRunningTime="2026-11-03 16:34:33.877581637 +0000 UTC m=+1359.702032911"
Nov 03 16:34:37.589430 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:37.589396    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:34:44.604170 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:44.604142    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:34:44.604570 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:44.604207    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:34:48.575875 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.575844    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"]
Nov 03 16:34:48.593987 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.593958    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.639815 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.626899    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"]
Nov 03 16:34:48.671133 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.671099    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dffeb6e9-03ca-4250-9032-bbe295bbf959-openshift-service-ca\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.671337 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.671148    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-metrics-certs\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.671520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.671496    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-maas-api-tls\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.671648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.671538    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts86m\" (UniqueName: \"kubernetes.io/projected/dffeb6e9-03ca-4250-9032-bbe295bbf959-kube-api-access-ts86m\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.672261 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.672246    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-06f127ec-serving-cert\""
Nov 03 16:34:48.773229 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.773198    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-maas-api-tls\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.773434 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.773240    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts86m\" (UniqueName: \"kubernetes.io/projected/dffeb6e9-03ca-4250-9032-bbe295bbf959-kube-api-access-ts86m\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.773434 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.773306    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dffeb6e9-03ca-4250-9032-bbe295bbf959-openshift-service-ca\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.773434 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.773338    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-metrics-certs\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.773434 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:34:48.773343    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-serving-cert: secret "maas-api-e2e-mt-06f127ec-serving-cert" not found
Nov 03 16:34:48.773434 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:34:48.773403    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-maas-api-tls podName:dffeb6e9-03ca-4250-9032-bbe295bbf959 nodeName:}" failed. No retries permitted until 2026-11-03 16:34:49.273387447 +0000 UTC m=+1375.097838713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-maas-api-tls") pod "maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2" (UID: "dffeb6e9-03ca-4250-9032-bbe295bbf959") : secret "maas-api-e2e-mt-06f127ec-serving-cert" not found
Nov 03 16:34:48.773994 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.773959    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dffeb6e9-03ca-4250-9032-bbe295bbf959-openshift-service-ca\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.775835 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.775811    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-metrics-certs\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:48.876652 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:48.875676    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts86m\" (UniqueName: \"kubernetes.io/projected/dffeb6e9-03ca-4250-9032-bbe295bbf959-kube-api-access-ts86m\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:49.308330 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:49.308234    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-maas-api-tls\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:49.344330 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:49.344297    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-maas-api-tls\") pod \"maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:49.543754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:49.538056    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:50.434515 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:50.434476    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"]
Nov 03 16:34:50.969864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:50.969829    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2" event={"ID":"dffeb6e9-03ca-4250-9032-bbe295bbf959","Type":"ContainerStarted","Data":"9dc0a3c9bcab013ada1c3388c9c2fe15ecf5082a82a55a960161d2b761fd775c"}
Nov 03 16:34:52.166330 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:52.166285    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2" event={"ID":"dffeb6e9-03ca-4250-9032-bbe295bbf959","Type":"ContainerStarted","Data":"aed836f0b869ff6dfb65e9f57ced813275d9e25032d2843c5a5bf2b30a639158"}
Nov 03 16:34:52.167285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:52.167262    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:34:53.674994 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.674926    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2" podStartSLOduration=4.777614151 podStartE2EDuration="5.674910064s" totalImagesPullingTime="897.295913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:34:52.335087232 +0000 UTC m=+1378.159538504" watchObservedRunningTime="2026-11-03 16:34:53.674910064 +0000 UTC m=+1379.499361377"
Nov 03 16:34:53.675432 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.675416    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"]
Nov 03 16:34:53.679853 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.679831    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.735520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.735104    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f88b986-40aa-4b9d-91e3-083b2a863599-openshift-service-ca\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.735520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.735301    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-maas-api-tls\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.735520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.735468    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47l56\" (UniqueName: \"kubernetes.io/projected/0f88b986-40aa-4b9d-91e3-083b2a863599-kube-api-access-47l56\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.735520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.735517    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-metrics-certs\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.740500 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.740473    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"]
Nov 03 16:34:53.773269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.773243    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e6def865-serving-cert\""
Nov 03 16:34:53.843303 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.843265    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-metrics-certs\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.843658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.843633    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f88b986-40aa-4b9d-91e3-083b2a863599-openshift-service-ca\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.843983 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.843961    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-maas-api-tls\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.844630 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:34:53.844175    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-serving-cert: secret "maas-api-e2e-migrate-e6def865-serving-cert" not found
Nov 03 16:34:53.844630 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:34:53.844357    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-maas-api-tls podName:0f88b986-40aa-4b9d-91e3-083b2a863599 nodeName:}" failed. No retries permitted until 2026-11-03 16:34:54.344329896 +0000 UTC m=+1380.168781159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-maas-api-tls") pod "maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz" (UID: "0f88b986-40aa-4b9d-91e3-083b2a863599") : secret "maas-api-e2e-migrate-e6def865-serving-cert" not found
Nov 03 16:34:53.844630 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.844419    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f88b986-40aa-4b9d-91e3-083b2a863599-openshift-service-ca\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.844863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.844714    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47l56\" (UniqueName: \"kubernetes.io/projected/0f88b986-40aa-4b9d-91e3-083b2a863599-kube-api-access-47l56\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.847309 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.847284    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-metrics-certs\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:53.958813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:53.956415    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47l56\" (UniqueName: \"kubernetes.io/projected/0f88b986-40aa-4b9d-91e3-083b2a863599-kube-api-access-47l56\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:54.401764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:54.387438    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-maas-api-tls\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:54.441761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:54.441706    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-maas-api-tls\") pod \"maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:54.639468 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:54.639432    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:55.078904 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:55.078870    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"]
Nov 03 16:34:55.103113 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:34:55.103065    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f88b986_40aa_4b9d_91e3_083b2a863599.slice/crio-872f1aab3c1f7e19de607bc404ecec98abbb17b7861ea08bc4178619243d4d7d WatchSource:0}: Error finding container 872f1aab3c1f7e19de607bc404ecec98abbb17b7861ea08bc4178619243d4d7d: Status 404 returned error can't find the container with id 872f1aab3c1f7e19de607bc404ecec98abbb17b7861ea08bc4178619243d4d7d
Nov 03 16:34:55.240390 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:55.240356    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz" event={"ID":"0f88b986-40aa-4b9d-91e3-083b2a863599","Type":"ContainerStarted","Data":"872f1aab3c1f7e19de607bc404ecec98abbb17b7861ea08bc4178619243d4d7d"}
Nov 03 16:34:56.251243 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:56.251206    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz" event={"ID":"0f88b986-40aa-4b9d-91e3-083b2a863599","Type":"ContainerStarted","Data":"f0d2d57956d8fcdc9989e06c5476bbec709edaa22148b95166b965431b2e3ed1"}
Nov 03 16:34:56.251693 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:56.251300    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:34:56.373424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:56.373364    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz" podStartSLOduration=2.360956251 podStartE2EDuration="3.373347884s" totalImagesPullingTime="1.012391633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:34:56.372742335 +0000 UTC m=+1382.197193601" watchObservedRunningTime="2026-11-03 16:34:56.373347884 +0000 UTC m=+1382.197799157"
Nov 03 16:34:58.181278 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:34:58.181243    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:35:00.881294 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:00.881264    5605 patch_prober.go:29] interesting pod/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 16:35:00.881743 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:00.881332    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 16:35:01.294842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.294803    5605 generic.go:356] "Generic (PLEG): container finished" podID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerID="9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b" exitCode=0
Nov 03 16:35:01.295013 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.294839    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" event={"ID":"374d3275-56a4-4abf-89e3-34e7ee57294c","Type":"ContainerDied","Data":"9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b"}
Nov 03 16:35:01.295403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.295380    5605 scope.go:122] "RemoveContainer" containerID="9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b"
Nov 03 16:35:01.296779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.296712    5605 generic.go:356] "Generic (PLEG): container finished" podID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerID="38077d001c7df2dd48d513f8988962f88b58fd1a7125bda0fb118f6099da05c1" exitCode=0
Nov 03 16:35:01.296881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.296754    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" event={"ID":"afeb5a90-adb0-4757-828a-1cdf537c1414","Type":"ContainerDied","Data":"38077d001c7df2dd48d513f8988962f88b58fd1a7125bda0fb118f6099da05c1"}
Nov 03 16:35:01.297150 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.297120    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:35:01.297188 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.297169    5605 scope.go:122] "RemoveContainer" containerID="38077d001c7df2dd48d513f8988962f88b58fd1a7125bda0fb118f6099da05c1"
Nov 03 16:35:01.297222 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.297191    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:35:01.297259 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.297227    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:35:01.298592 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.298566    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:35:01.298654 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.298629    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:35:01.298692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:01.298661    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:35:02.265698 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:02.265662    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:35:02.343393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:02.343348    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" event={"ID":"afeb5a90-adb0-4757-828a-1cdf537c1414","Type":"ContainerStarted","Data":"be7e073bed06b26afcfa9b2115a2d9c957d525ee34c31587b53e2cbcc234b508"}
Nov 03 16:35:02.343760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:02.343700    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" event={"ID":"374d3275-56a4-4abf-89e3-34e7ee57294c","Type":"ContainerStarted","Data":"c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23"}
Nov 03 16:35:02.655632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:02.655598    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:35:02.655853 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:02.655786    5605 patch_prober.go:29] interesting pod/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 16:35:02.655920 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:02.655850    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 16:35:02.843150 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:02.843104    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:35:02.844004 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:02.843979    5605 patch_prober.go:29] interesting pod/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 16:35:02.844059 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:02.844036    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 16:35:03.441075 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:35:03.441032    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:35:03.655617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:03.655587    5605 patch_prober.go:29] interesting pod/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 16:35:03.655886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:03.655857    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 16:35:03.851438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:03.851392    5605 patch_prober.go:29] interesting pod/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 16:35:03.851713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:03.851687    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 16:35:04.655250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:04.655217    5605 patch_prober.go:29] interesting pod/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 16:35:04.655797 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:04.655285    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 16:35:04.842141 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:04.842109    5605 patch_prober.go:29] interesting pod/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 16:35:04.842308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:04.842159    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 16:35:05.655469 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:05.655438    5605 patch_prober.go:29] interesting pod/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 16:35:05.655995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:05.655498    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 16:35:05.842346 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:05.842313    5605 patch_prober.go:29] interesting pod/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 16:35:05.842562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:05.842367    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 16:35:06.216855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:06.216821    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:35:06.655690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:06.655653    5605 patch_prober.go:29] interesting pod/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 16:35:06.656114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:06.655744    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 16:35:06.842640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:06.842601    5605 patch_prober.go:29] interesting pod/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 16:35:06.842828 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:06.842662    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 16:35:07.656331 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:07.656291    5605 patch_prober.go:29] interesting pod/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 16:35:07.656863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:07.656361    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 16:35:07.846930 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:07.846903    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:35:07.848009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:07.847988    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:35:08.660484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:08.660455    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:35:08.660915 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:08.660896    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:35:08.661494 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:08.661473    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:35:18.113504 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:18.113472    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"]
Nov 03 16:35:18.165089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:18.165054    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-20ad82e1-jhgqm"]
Nov 03 16:35:18.310422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:35:18.310388    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf97655c-9734-415b-9c9e-a8353645d126" path="/var/lib/kubelet/pods/cf97655c-9734-415b-9c9e-a8353645d126/volumes"
Nov 03 16:35:33.394236 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:35:33.394199    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:36:03.393326 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:36:03.393276    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:36:27.261336 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.261305    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"]
Nov 03 16:36:27.261933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.261535    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t" podUID="b9196ae2-3beb-4687-a400-98a612a4988d" containerName="maas-api" containerID="cri-o://bfb68dace4fc7a70dc3885bd0da9aa5ca5f3911f039c431df88fc314ccc5ba36" gracePeriod=30
Nov 03 16:36:27.656046 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.647501    5605 generic.go:356] "Generic (PLEG): container finished" podID="b9196ae2-3beb-4687-a400-98a612a4988d" containerID="bfb68dace4fc7a70dc3885bd0da9aa5ca5f3911f039c431df88fc314ccc5ba36" exitCode=0
Nov 03 16:36:27.656046 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.647659    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t" event={"ID":"b9196ae2-3beb-4687-a400-98a612a4988d","Type":"ContainerDied","Data":"bfb68dace4fc7a70dc3885bd0da9aa5ca5f3911f039c431df88fc314ccc5ba36"}
Nov 03 16:36:27.777805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.771444    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:36:27.950811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.950015    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9196ae2-3beb-4687-a400-98a612a4988d-kube-api-access-997dk\" (UniqueName: \"kubernetes.io/projected/b9196ae2-3beb-4687-a400-98a612a4988d-kube-api-access-997dk\") pod \"b9196ae2-3beb-4687-a400-98a612a4988d\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") "
Nov 03 16:36:27.950811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.950076    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9196ae2-3beb-4687-a400-98a612a4988d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9196ae2-3beb-4687-a400-98a612a4988d-openshift-service-ca\") pod \"b9196ae2-3beb-4687-a400-98a612a4988d\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") "
Nov 03 16:36:27.950811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.950158    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-maas-api-tls\") pod \"b9196ae2-3beb-4687-a400-98a612a4988d\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") "
Nov 03 16:36:27.950811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.950253    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-metrics-certs\") pod \"b9196ae2-3beb-4687-a400-98a612a4988d\" (UID: \"b9196ae2-3beb-4687-a400-98a612a4988d\") "
Nov 03 16:36:27.958592 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.956310    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9196ae2-3beb-4687-a400-98a612a4988d-openshift-service-ca" pod "b9196ae2-3beb-4687-a400-98a612a4988d" (UID: "b9196ae2-3beb-4687-a400-98a612a4988d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:36:27.965877 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.964005    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-maas-api-tls" pod "b9196ae2-3beb-4687-a400-98a612a4988d" (UID: "b9196ae2-3beb-4687-a400-98a612a4988d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:36:27.966606 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.966574    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-metrics-certs" pod "b9196ae2-3beb-4687-a400-98a612a4988d" (UID: "b9196ae2-3beb-4687-a400-98a612a4988d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:36:27.967030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:27.967009    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9196ae2-3beb-4687-a400-98a612a4988d-kube-api-access-997dk" pod "b9196ae2-3beb-4687-a400-98a612a4988d" (UID: "b9196ae2-3beb-4687-a400-98a612a4988d"). InnerVolumeSpecName "kube-api-access-997dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:36:28.052739 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:28.052690    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-997dk\" (UniqueName: \"kubernetes.io/projected/b9196ae2-3beb-4687-a400-98a612a4988d-kube-api-access-997dk\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:28.052917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:28.052759    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9196ae2-3beb-4687-a400-98a612a4988d-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:28.052917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:28.052771    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:28.052917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:28.052779    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9196ae2-3beb-4687-a400-98a612a4988d-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:28.655242 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:28.655209    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"
Nov 03 16:36:28.655764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:28.655251    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t" event={"ID":"b9196ae2-3beb-4687-a400-98a612a4988d","Type":"ContainerDied","Data":"dfa4df8051a1d3a142f9fbc088a83bb20ab2590c69a8d3e3bed2d69b00c042dc"}
Nov 03 16:36:28.655764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:28.655311    5605 scope.go:122] "RemoveContainer" containerID="bfb68dace4fc7a70dc3885bd0da9aa5ca5f3911f039c431df88fc314ccc5ba36"
Nov 03 16:36:28.833015 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:28.832986    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"]
Nov 03 16:36:28.883942 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:28.883911    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e998bdb4-5b9676cff8-mgk7t"]
Nov 03 16:36:29.519534 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:29.519501    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"]
Nov 03 16:36:29.519794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:29.519715    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy" containerID="cri-o://be7e073bed06b26afcfa9b2115a2d9c957d525ee34c31587b53e2cbcc234b508" gracePeriod=30
Nov 03 16:36:30.310218 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:30.310188    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9196ae2-3beb-4687-a400-98a612a4988d" path="/var/lib/kubelet/pods/b9196ae2-3beb-4687-a400-98a612a4988d/volumes"
Nov 03 16:36:33.394706 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:36:33.394668    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:36:34.807181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:34.802948    5605 generic.go:356] "Generic (PLEG): container finished" podID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerID="be7e073bed06b26afcfa9b2115a2d9c957d525ee34c31587b53e2cbcc234b508" exitCode=0
Nov 03 16:36:34.807181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:34.803100    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" event={"ID":"afeb5a90-adb0-4757-828a-1cdf537c1414","Type":"ContainerDied","Data":"be7e073bed06b26afcfa9b2115a2d9c957d525ee34c31587b53e2cbcc234b508"}
Nov 03 16:36:34.807181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:34.803149    5605 scope.go:122] "RemoveContainer" containerID="38077d001c7df2dd48d513f8988962f88b58fd1a7125bda0fb118f6099da05c1"
Nov 03 16:36:34.906535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:34.906506    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:36:35.066669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.066585    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-socket\") pod \"afeb5a90-adb0-4757-828a-1cdf537c1414\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") "
Nov 03 16:36:35.066865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.066705    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-envoy\") pod \"afeb5a90-adb0-4757-828a-1cdf537c1414\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") "
Nov 03 16:36:35.066865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.066787    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-credential-socket\") pod \"afeb5a90-adb0-4757-828a-1cdf537c1414\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") "
Nov 03 16:36:35.066865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.066831    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-token\" (UniqueName: \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-token\") pod \"afeb5a90-adb0-4757-828a-1cdf537c1414\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") "
Nov 03 16:36:35.066865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.066852    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-socket" pod "afeb5a90-adb0-4757-828a-1cdf537c1414" (UID: "afeb5a90-adb0-4757-828a-1cdf537c1414"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:36:35.067093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.066858    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-data\") pod \"afeb5a90-adb0-4757-828a-1cdf537c1414\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") "
Nov 03 16:36:35.067093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.067027    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-podinfo\") pod \"afeb5a90-adb0-4757-828a-1cdf537c1414\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") "
Nov 03 16:36:35.067183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.067100    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-data" pod "afeb5a90-adb0-4757-828a-1cdf537c1414" (UID: "afeb5a90-adb0-4757-828a-1cdf537c1414"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:36:35.067183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.067120    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-credential-socket" pod "afeb5a90-adb0-4757-828a-1cdf537c1414" (UID: "afeb5a90-adb0-4757-828a-1cdf537c1414"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:36:35.067271 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.067200    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-kube-api-access-srtv2\" (UniqueName: \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-kube-api-access-srtv2\") pod \"afeb5a90-adb0-4757-828a-1cdf537c1414\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") "
Nov 03 16:36:35.067271 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.067230    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/afeb5a90-adb0-4757-828a-1cdf537c1414-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afeb5a90-adb0-4757-828a-1cdf537c1414-istiod-ca-cert\") pod \"afeb5a90-adb0-4757-828a-1cdf537c1414\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") "
Nov 03 16:36:35.067381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.067274    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-certs\") pod \"afeb5a90-adb0-4757-828a-1cdf537c1414\" (UID: \"afeb5a90-adb0-4757-828a-1cdf537c1414\") "
Nov 03 16:36:35.067948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.067864    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-certs" pod "afeb5a90-adb0-4757-828a-1cdf537c1414" (UID: "afeb5a90-adb0-4757-828a-1cdf537c1414"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:36:35.067948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.067899    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afeb5a90-adb0-4757-828a-1cdf537c1414-istiod-ca-cert" pod "afeb5a90-adb0-4757-828a-1cdf537c1414" (UID: "afeb5a90-adb0-4757-828a-1cdf537c1414"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:36:35.068154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.068133    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:35.068221 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.068161    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:35.068221 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.068177    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afeb5a90-adb0-4757-828a-1cdf537c1414-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:35.068221 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.068190    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:35.068221 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.068203    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:35.069227 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.069200    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-envoy" pod "afeb5a90-adb0-4757-828a-1cdf537c1414" (UID: "afeb5a90-adb0-4757-828a-1cdf537c1414"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:36:35.069495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.069477    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-kube-api-access-srtv2" pod "afeb5a90-adb0-4757-828a-1cdf537c1414" (UID: "afeb5a90-adb0-4757-828a-1cdf537c1414"). InnerVolumeSpecName "kube-api-access-srtv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:36:35.069700 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.069676    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-podinfo" pod "afeb5a90-adb0-4757-828a-1cdf537c1414" (UID: "afeb5a90-adb0-4757-828a-1cdf537c1414"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:36:35.069788 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.069680    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-token" pod "afeb5a90-adb0-4757-828a-1cdf537c1414" (UID: "afeb5a90-adb0-4757-828a-1cdf537c1414"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:36:35.138353 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.138308    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"]
Nov 03 16:36:35.169667 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.169628    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:35.169667 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.169659    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:35.169667 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.169669    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afeb5a90-adb0-4757-828a-1cdf537c1414-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:35.169957 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.169679    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srtv2\" (UniqueName: \"kubernetes.io/projected/afeb5a90-adb0-4757-828a-1cdf537c1414-kube-api-access-srtv2\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:35.278463 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.278429    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"]
Nov 03 16:36:35.895922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.895882    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6" event={"ID":"afeb5a90-adb0-4757-828a-1cdf537c1414","Type":"ContainerDied","Data":"8dc06b51338ee9e5df5283f3c8de43268d79daea55645db160ceff80d7222694"}
Nov 03 16:36:35.896607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.896583    5605 scope.go:122] "RemoveContainer" containerID="be7e073bed06b26afcfa9b2115a2d9c957d525ee34c31587b53e2cbcc234b508"
Nov 03 16:36:35.896934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:35.896916    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"
Nov 03 16:36:36.075433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:36.075398    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"]
Nov 03 16:36:36.080920 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:36.080891    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e998bdb4-openshift-default-6f9cdcfd5f-dcwp6"]
Nov 03 16:36:36.311567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:36.311476    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" path="/var/lib/kubelet/pods/afeb5a90-adb0-4757-828a-1cdf537c1414/volumes"
Nov 03 16:36:38.720090 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:38.720059    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"]
Nov 03 16:36:38.720577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:38.720311    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq" podUID="6f868573-448d-4ddf-8043-aa2c181800fd" containerName="maas-api" containerID="cri-o://33a83c3d10c5e3e1fa2de2437f2ddad48ea9e1e78e7f436c3b9ae727b23ccf73" gracePeriod=30
Nov 03 16:36:39.117106 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.111710    5605 generic.go:356] "Generic (PLEG): container finished" podID="6f868573-448d-4ddf-8043-aa2c181800fd" containerID="33a83c3d10c5e3e1fa2de2437f2ddad48ea9e1e78e7f436c3b9ae727b23ccf73" exitCode=0
Nov 03 16:36:39.117106 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.116384    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq" event={"ID":"6f868573-448d-4ddf-8043-aa2c181800fd","Type":"ContainerDied","Data":"33a83c3d10c5e3e1fa2de2437f2ddad48ea9e1e78e7f436c3b9ae727b23ccf73"}
Nov 03 16:36:39.180325 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.179944    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:36:39.332801 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.321690    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-maas-api-tls\") pod \"6f868573-448d-4ddf-8043-aa2c181800fd\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") "
Nov 03 16:36:39.333051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.333028    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f868573-448d-4ddf-8043-aa2c181800fd-kube-api-access-gcf8s\" (UniqueName: \"kubernetes.io/projected/6f868573-448d-4ddf-8043-aa2c181800fd-kube-api-access-gcf8s\") pod \"6f868573-448d-4ddf-8043-aa2c181800fd\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") "
Nov 03 16:36:39.333647 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.333622    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-metrics-certs\") pod \"6f868573-448d-4ddf-8043-aa2c181800fd\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") "
Nov 03 16:36:39.334396 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.334376    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f868573-448d-4ddf-8043-aa2c181800fd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f868573-448d-4ddf-8043-aa2c181800fd-openshift-service-ca\") pod \"6f868573-448d-4ddf-8043-aa2c181800fd\" (UID: \"6f868573-448d-4ddf-8043-aa2c181800fd\") "
Nov 03 16:36:39.335088 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.334844    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-maas-api-tls" pod "6f868573-448d-4ddf-8043-aa2c181800fd" (UID: "6f868573-448d-4ddf-8043-aa2c181800fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:36:39.335232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.334971    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f868573-448d-4ddf-8043-aa2c181800fd-openshift-service-ca" pod "6f868573-448d-4ddf-8043-aa2c181800fd" (UID: "6f868573-448d-4ddf-8043-aa2c181800fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:36:39.336111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.336088    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f868573-448d-4ddf-8043-aa2c181800fd-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:39.336248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.336235    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:39.356134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.351370    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f868573-448d-4ddf-8043-aa2c181800fd-kube-api-access-gcf8s" pod "6f868573-448d-4ddf-8043-aa2c181800fd" (UID: "6f868573-448d-4ddf-8043-aa2c181800fd"). InnerVolumeSpecName "kube-api-access-gcf8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:36:39.356134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.355820    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-metrics-certs" pod "6f868573-448d-4ddf-8043-aa2c181800fd" (UID: "6f868573-448d-4ddf-8043-aa2c181800fd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:36:39.446660 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.446104    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcf8s\" (UniqueName: \"kubernetes.io/projected/6f868573-448d-4ddf-8043-aa2c181800fd-kube-api-access-gcf8s\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:39.446660 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:39.446146    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f868573-448d-4ddf-8043-aa2c181800fd-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:40.125552 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.125516    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"
Nov 03 16:36:40.125552 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.125544    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq" event={"ID":"6f868573-448d-4ddf-8043-aa2c181800fd","Type":"ContainerDied","Data":"512a995bd5bb696b17ad0fdf3a684a9f467a64e383530a48a35c42380d97a058"}
Nov 03 16:36:40.126157 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.125606    5605 scope.go:122] "RemoveContainer" containerID="33a83c3d10c5e3e1fa2de2437f2ddad48ea9e1e78e7f436c3b9ae727b23ccf73"
Nov 03 16:36:40.139163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.139118    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq" podUID="09f0a14b-47aa-4887-855d-9b3b1600eb4d" containerName="payload-processing" containerID="cri-o://00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716" gracePeriod=25
Nov 03 16:36:40.279291 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.279253    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl" podUID="700a9035-03d8-4cb2-a9e3-c0718f114297" containerName="payload-pre-processing" containerID="cri-o://c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93" gracePeriod=25
Nov 03 16:36:40.312092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.312061    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"]
Nov 03 16:36:40.357767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.357322    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1b774106-5c98c6db77-g7fgq"]
Nov 03 16:36:40.439079 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.439044    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:36:40.485863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.485831    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/09f0a14b-47aa-4887-855d-9b3b1600eb4d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09f0a14b-47aa-4887-855d-9b3b1600eb4d-plugins-config-volume\") pod \"09f0a14b-47aa-4887-855d-9b3b1600eb4d\" (UID: \"09f0a14b-47aa-4887-855d-9b3b1600eb4d\") "
Nov 03 16:36:40.486096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.486075    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09f0a14b-47aa-4887-855d-9b3b1600eb4d-kube-api-access-q285d\" (UniqueName: \"kubernetes.io/projected/09f0a14b-47aa-4887-855d-9b3b1600eb4d-kube-api-access-q285d\") pod \"09f0a14b-47aa-4887-855d-9b3b1600eb4d\" (UID: \"09f0a14b-47aa-4887-855d-9b3b1600eb4d\") "
Nov 03 16:36:40.486561 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.486267    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09f0a14b-47aa-4887-855d-9b3b1600eb4d-plugins-config-volume" pod "09f0a14b-47aa-4887-855d-9b3b1600eb4d" (UID: "09f0a14b-47aa-4887-855d-9b3b1600eb4d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:36:40.486785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.486763    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09f0a14b-47aa-4887-855d-9b3b1600eb4d-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:40.488936 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.488914    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f0a14b-47aa-4887-855d-9b3b1600eb4d-kube-api-access-q285d" pod "09f0a14b-47aa-4887-855d-9b3b1600eb4d" (UID: "09f0a14b-47aa-4887-855d-9b3b1600eb4d"). InnerVolumeSpecName "kube-api-access-q285d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:36:40.553160 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.553114    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:36:40.591785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.588658    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q285d\" (UniqueName: \"kubernetes.io/projected/09f0a14b-47aa-4887-855d-9b3b1600eb4d-kube-api-access-q285d\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:40.777874 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.774624    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/700a9035-03d8-4cb2-a9e3-c0718f114297-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/700a9035-03d8-4cb2-a9e3-c0718f114297-plugins-config-volume\") pod \"700a9035-03d8-4cb2-a9e3-c0718f114297\" (UID: \"700a9035-03d8-4cb2-a9e3-c0718f114297\") "
Nov 03 16:36:40.794293 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.784318    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/700a9035-03d8-4cb2-a9e3-c0718f114297-plugins-config-volume" pod "700a9035-03d8-4cb2-a9e3-c0718f114297" (UID: "700a9035-03d8-4cb2-a9e3-c0718f114297"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:36:40.794293 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.791138    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/700a9035-03d8-4cb2-a9e3-c0718f114297-kube-api-access-7k2zd\" (UniqueName: \"kubernetes.io/projected/700a9035-03d8-4cb2-a9e3-c0718f114297-kube-api-access-7k2zd\") pod \"700a9035-03d8-4cb2-a9e3-c0718f114297\" (UID: \"700a9035-03d8-4cb2-a9e3-c0718f114297\") "
Nov 03 16:36:40.794293 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.792254    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/700a9035-03d8-4cb2-a9e3-c0718f114297-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:40.803541 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.803484    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/700a9035-03d8-4cb2-a9e3-c0718f114297-kube-api-access-7k2zd" pod "700a9035-03d8-4cb2-a9e3-c0718f114297" (UID: "700a9035-03d8-4cb2-a9e3-c0718f114297"). InnerVolumeSpecName "kube-api-access-7k2zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:36:40.893374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:40.893343    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k2zd\" (UniqueName: \"kubernetes.io/projected/700a9035-03d8-4cb2-a9e3-c0718f114297-kube-api-access-7k2zd\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:41.133911 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.133874    5605 generic.go:356] "Generic (PLEG): container finished" podID="09f0a14b-47aa-4887-855d-9b3b1600eb4d" containerID="00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716" exitCode=0
Nov 03 16:36:41.134374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.133934    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"
Nov 03 16:36:41.134374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.133959    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq" event={"ID":"09f0a14b-47aa-4887-855d-9b3b1600eb4d","Type":"ContainerDied","Data":"00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716"}
Nov 03 16:36:41.134374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.134070    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq" event={"ID":"09f0a14b-47aa-4887-855d-9b3b1600eb4d","Type":"ContainerDied","Data":"eb3ab9d5d6d77a54c01226616578330879392e0c8f20824f0966fa1e14d7c9ae"}
Nov 03 16:36:41.134374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.134105    5605 scope.go:122] "RemoveContainer" containerID="00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716"
Nov 03 16:36:41.136945 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.136916    5605 generic.go:356] "Generic (PLEG): container finished" podID="700a9035-03d8-4cb2-a9e3-c0718f114297" containerID="c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93" exitCode=0
Nov 03 16:36:41.137089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.137030    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"
Nov 03 16:36:41.137089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.137061    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl" event={"ID":"700a9035-03d8-4cb2-a9e3-c0718f114297","Type":"ContainerDied","Data":"c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93"}
Nov 03 16:36:41.137197 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.137114    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl" event={"ID":"700a9035-03d8-4cb2-a9e3-c0718f114297","Type":"ContainerDied","Data":"63f42599b00675df2dbf82c347691ab2f2f2782cba8c25f4d4e54eb36a3ddcf8"}
Nov 03 16:36:41.146144 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.146123    5605 scope.go:122] "RemoveContainer" containerID="00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716"
Nov 03 16:36:41.146428 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:36:41.146403    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716\": container with ID starting with 00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716 not found: ID does not exist" containerID="00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716"
Nov 03 16:36:41.146495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.146442    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716"} err="failed to get container status \"00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716\": rpc error: code = NotFound desc = could not find container \"00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716\": container with ID starting with 00c960c413ef3b636f0d3d2a2d1f70c887bd87bb4731af8c66b075821d6d8716 not found: ID does not exist"
Nov 03 16:36:41.146495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.146469    5605 scope.go:122] "RemoveContainer" containerID="c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93"
Nov 03 16:36:41.156055 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.156037    5605 scope.go:122] "RemoveContainer" containerID="c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93"
Nov 03 16:36:41.156301 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:36:41.156279    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93\": container with ID starting with c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93 not found: ID does not exist" containerID="c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93"
Nov 03 16:36:41.156360 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.156308    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93"} err="failed to get container status \"c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93\": rpc error: code = NotFound desc = could not find container \"c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93\": container with ID starting with c9c0c536a24168274f02046d4c5abcf148731a58c94920d06d573acadb1d0b93 not found: ID does not exist"
Nov 03 16:36:41.312935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.312905    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"]
Nov 03 16:36:41.318140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.318114    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-161f6056-6f99d7f9bc-2rhdl"]
Nov 03 16:36:41.435773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.435638    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"]
Nov 03 16:36:41.439900 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:41.439875    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-161f6056-7b9cc86dd5-lxjfq"]
Nov 03 16:36:42.310373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:42.310341    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09f0a14b-47aa-4887-855d-9b3b1600eb4d" path="/var/lib/kubelet/pods/09f0a14b-47aa-4887-855d-9b3b1600eb4d/volumes"
Nov 03 16:36:42.310817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:42.310802    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f868573-448d-4ddf-8043-aa2c181800fd" path="/var/lib/kubelet/pods/6f868573-448d-4ddf-8043-aa2c181800fd/volumes"
Nov 03 16:36:42.311197 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:42.311183    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="700a9035-03d8-4cb2-a9e3-c0718f114297" path="/var/lib/kubelet/pods/700a9035-03d8-4cb2-a9e3-c0718f114297/volumes"
Nov 03 16:36:42.617769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:42.617736    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"]
Nov 03 16:36:42.617982 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:42.617961    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy" containerID="cri-o://c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23" gracePeriod=30
Nov 03 16:36:44.018839 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.015831    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"]
Nov 03 16:36:44.094760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.094694    5605 cpu_manager.go:414] "containerMap: removing container" podUID="700a9035-03d8-4cb2-a9e3-c0718f114297" containerName="payload-pre-processing"
Nov 03 16:36:44.095061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.094785    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="700a9035-03d8-4cb2-a9e3-c0718f114297" containerName="payload-pre-processing"
Nov 03 16:36:44.095061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.094806    5605 cpu_manager.go:414] "containerMap: removing container" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy"
Nov 03 16:36:44.095061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.094814    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy"
Nov 03 16:36:44.095061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.094850    5605 cpu_manager.go:414] "containerMap: removing container" podUID="b9196ae2-3beb-4687-a400-98a612a4988d" containerName="maas-api"
Nov 03 16:36:44.095061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.094858    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9196ae2-3beb-4687-a400-98a612a4988d" containerName="maas-api"
Nov 03 16:36:44.095061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.094871    5605 cpu_manager.go:414] "containerMap: removing container" podUID="09f0a14b-47aa-4887-855d-9b3b1600eb4d" containerName="payload-processing"
Nov 03 16:36:44.095061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.094878    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09f0a14b-47aa-4887-855d-9b3b1600eb4d" containerName="payload-processing"
Nov 03 16:36:44.095061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.094910    5605 cpu_manager.go:414] "containerMap: removing container" podUID="6f868573-448d-4ddf-8043-aa2c181800fd" containerName="maas-api"
Nov 03 16:36:44.095061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.094923    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f868573-448d-4ddf-8043-aa2c181800fd" containerName="maas-api"
Nov 03 16:36:44.135162 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.135124    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy"
Nov 03 16:36:44.135428 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.135408    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f868573-448d-4ddf-8043-aa2c181800fd" containerName="maas-api"
Nov 03 16:36:44.138804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.135565    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="700a9035-03d8-4cb2-a9e3-c0718f114297" containerName="payload-pre-processing"
Nov 03 16:36:44.148821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.148784    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy"
Nov 03 16:36:44.149069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.149055    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9196ae2-3beb-4687-a400-98a612a4988d" containerName="maas-api"
Nov 03 16:36:44.149159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.149150    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="09f0a14b-47aa-4887-855d-9b3b1600eb4d" containerName="payload-processing"
Nov 03 16:36:44.215484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.215442    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"]
Nov 03 16:36:44.215676 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.215580    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.233719 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.233686    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-token\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.233984 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.233958    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8c082b7-7db2-4f35-a62d-51750ca323ab-istiod-ca-cert\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.234528 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.234489    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-envoy\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.234652 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.234627    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-credential-socket\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.234815 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.234694    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-podinfo\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.234902 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.234877    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-data\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.234961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.234946    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-certs\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.235110 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.235094    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-socket\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.235171 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.235146    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmv64\" (UniqueName: \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-kube-api-access-dmv64\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.354217    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b70a8a53-openshift-default-dockercfg-gvx67\""
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.354818    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-credential-socket\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.354873    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-podinfo\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.354974    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-data\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.355037    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-certs\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.355182    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-socket\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.355227    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmv64\" (UniqueName: \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-kube-api-access-dmv64\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.355366    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-token\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.355401    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8c082b7-7db2-4f35-a62d-51750ca323ab-istiod-ca-cert\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.355430    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-envoy\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.356223    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-credential-socket\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.356463    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-socket\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.356694    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-data\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.367849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.356943    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-certs\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.402993 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.376237    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8c082b7-7db2-4f35-a62d-51750ca323ab-istiod-ca-cert\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.402993 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.401930    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-podinfo\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.410748 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.407946    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-envoy\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.424754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.421185    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmv64\" (UniqueName: \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-kube-api-access-dmv64\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.424754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.423851    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-token\") pod \"e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") " pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.533434 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.533367    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:44.972245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.972215    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"]
Nov 03 16:36:44.972554 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:36:44.972529    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c082b7_7db2_4f35_a62d_51750ca323ab.slice/crio-63ffce6e9f206305364a1c2f132f1d866d6c51c25007028fad97deb693c60bc5 WatchSource:0}: Error finding container 63ffce6e9f206305364a1c2f132f1d866d6c51c25007028fad97deb693c60bc5: Status 404 returned error can't find the container with id 63ffce6e9f206305364a1c2f132f1d866d6c51c25007028fad97deb693c60bc5
Nov 03 16:36:44.975128 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.975092    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:36:44.975190 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.975175    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:36:44.975244 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:44.975227    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:36:45.203446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.203411    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" event={"ID":"a8c082b7-7db2-4f35-a62d-51750ca323ab","Type":"ContainerStarted","Data":"3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e"}
Nov 03 16:36:45.203941 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.203477    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" event={"ID":"a8c082b7-7db2-4f35-a62d-51750ca323ab","Type":"ContainerStarted","Data":"63ffce6e9f206305364a1c2f132f1d866d6c51c25007028fad97deb693c60bc5"}
Nov 03 16:36:45.330384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.330288    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" podStartSLOduration=2.330272368 podStartE2EDuration="2.330272368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:36:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:36:45.328277818 +0000 UTC m=+1491.152729092" watchObservedRunningTime="2026-11-03 16:36:45.330272368 +0000 UTC m=+1491.154723639"
Nov 03 16:36:45.543751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.537864    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:45.624995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.624966    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"]
Nov 03 16:36:45.625597 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.625581    5605 cpu_manager.go:414] "containerMap: removing container" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy"
Nov 03 16:36:45.625651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.625600    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afeb5a90-adb0-4757-828a-1cdf537c1414" containerName="istio-proxy"
Nov 03 16:36:45.629248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.629230    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:45.689090 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.689065    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-9f2868-serving-cert\""
Nov 03 16:36:45.719269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.719238    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"]
Nov 03 16:36:45.784861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.784821    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:45.785058 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.784975    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:45.785058 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.785006    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxn9s\" (UniqueName: \"kubernetes.io/projected/1a4e86aa-2c69-4803-994f-99c51afec557-kube-api-access-gxn9s\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:45.785185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.785096    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a4e86aa-2c69-4803-994f-99c51afec557-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:45.917388 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.917281    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:45.917564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.917445    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:45.917564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.917483    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxn9s\" (UniqueName: \"kubernetes.io/projected/1a4e86aa-2c69-4803-994f-99c51afec557-kube-api-access-gxn9s\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:45.917564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.917552    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a4e86aa-2c69-4803-994f-99c51afec557-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:45.918478 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.918452    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a4e86aa-2c69-4803-994f-99c51afec557-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:45.918599 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:36:45.918567    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-serving-cert: secret "maas-api-e2e-shared-a-w2-9f2868-serving-cert" not found
Nov 03 16:36:45.918660 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:36:45.918627    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-maas-api-tls podName:1a4e86aa-2c69-4803-994f-99c51afec557 nodeName:}" failed. No retries permitted until 2026-11-03 16:36:46.418606935 +0000 UTC m=+1492.243058198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-maas-api-tls") pod "maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd" (UID: "1a4e86aa-2c69-4803-994f-99c51afec557") : secret "maas-api-e2e-shared-a-w2-9f2868-serving-cert" not found
Nov 03 16:36:45.932649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.929870    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:46.002756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:45.997312    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxn9s\" (UniqueName: \"kubernetes.io/projected/1a4e86aa-2c69-4803-994f-99c51afec557-kube-api-access-gxn9s\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:46.426342 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:46.426307    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:46.428858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:46.428835    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:46.539588 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:46.539560    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:46.543858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:46.543832    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:46.931639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:46.931607    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"]
Nov 03 16:36:46.932824 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:36:46.932790    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4e86aa_2c69_4803_994f_99c51afec557.slice/crio-7ba9ae896cf71dfb4d581c450ac02967d12e18160b8eb99a74b12d30c7ee7a12 WatchSource:0}: Error finding container 7ba9ae896cf71dfb4d581c450ac02967d12e18160b8eb99a74b12d30c7ee7a12: Status 404 returned error can't find the container with id 7ba9ae896cf71dfb4d581c450ac02967d12e18160b8eb99a74b12d30c7ee7a12
Nov 03 16:36:46.934464 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:46.934440    5605 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 16:36:47.220774 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.220678    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd" event={"ID":"1a4e86aa-2c69-4803-994f-99c51afec557","Type":"ContainerStarted","Data":"7ba9ae896cf71dfb4d581c450ac02967d12e18160b8eb99a74b12d30c7ee7a12"}
Nov 03 16:36:47.220972 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.220954    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:47.222034 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.222009    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:36:47.932836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.932809    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:36:47.952842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.952813    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-data\") pod \"374d3275-56a4-4abf-89e3-34e7ee57294c\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") "
Nov 03 16:36:47.952974 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.952909    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-token\" (UniqueName: \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-token\") pod \"374d3275-56a4-4abf-89e3-34e7ee57294c\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") "
Nov 03 16:36:47.952974 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.952965    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-socket\") pod \"374d3275-56a4-4abf-89e3-34e7ee57294c\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") "
Nov 03 16:36:47.953092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.953028    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-kube-api-access-88ftj\" (UniqueName: \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-kube-api-access-88ftj\") pod \"374d3275-56a4-4abf-89e3-34e7ee57294c\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") "
Nov 03 16:36:47.953092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.953060    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-data" pod "374d3275-56a4-4abf-89e3-34e7ee57294c" (UID: "374d3275-56a4-4abf-89e3-34e7ee57294c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:36:47.953092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.953072    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-certs\") pod \"374d3275-56a4-4abf-89e3-34e7ee57294c\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") "
Nov 03 16:36:47.953239 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.953190    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-podinfo\") pod \"374d3275-56a4-4abf-89e3-34e7ee57294c\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") "
Nov 03 16:36:47.953321 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.953303    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-credential-socket\") pod \"374d3275-56a4-4abf-89e3-34e7ee57294c\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") "
Nov 03 16:36:47.953385 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.953317    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-certs" pod "374d3275-56a4-4abf-89e3-34e7ee57294c" (UID: "374d3275-56a4-4abf-89e3-34e7ee57294c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:36:47.953385 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.953327    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-socket" pod "374d3275-56a4-4abf-89e3-34e7ee57294c" (UID: "374d3275-56a4-4abf-89e3-34e7ee57294c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:36:47.953385 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.953369    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-envoy\") pod \"374d3275-56a4-4abf-89e3-34e7ee57294c\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") "
Nov 03 16:36:47.953543 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.953407    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/374d3275-56a4-4abf-89e3-34e7ee57294c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/374d3275-56a4-4abf-89e3-34e7ee57294c-istiod-ca-cert\") pod \"374d3275-56a4-4abf-89e3-34e7ee57294c\" (UID: \"374d3275-56a4-4abf-89e3-34e7ee57294c\") "
Nov 03 16:36:47.953636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.953614    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-credential-socket" pod "374d3275-56a4-4abf-89e3-34e7ee57294c" (UID: "374d3275-56a4-4abf-89e3-34e7ee57294c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:36:47.954283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.954206    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374d3275-56a4-4abf-89e3-34e7ee57294c-istiod-ca-cert" pod "374d3275-56a4-4abf-89e3-34e7ee57294c" (UID: "374d3275-56a4-4abf-89e3-34e7ee57294c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:36:47.954539 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.954500    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:47.954539 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.954529    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:47.954692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.954544    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:47.954692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.954557    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:47.955765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.955696    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-envoy" pod "374d3275-56a4-4abf-89e3-34e7ee57294c" (UID: "374d3275-56a4-4abf-89e3-34e7ee57294c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:36:47.955765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.955703    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-kube-api-access-88ftj" pod "374d3275-56a4-4abf-89e3-34e7ee57294c" (UID: "374d3275-56a4-4abf-89e3-34e7ee57294c"). InnerVolumeSpecName "kube-api-access-88ftj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:36:47.956021 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.955996    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-token" pod "374d3275-56a4-4abf-89e3-34e7ee57294c" (UID: "374d3275-56a4-4abf-89e3-34e7ee57294c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:36:47.956100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:47.956042    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-podinfo" pod "374d3275-56a4-4abf-89e3-34e7ee57294c" (UID: "374d3275-56a4-4abf-89e3-34e7ee57294c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:36:48.064769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.061670    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:48.064769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.061712    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:48.064769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.061745    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/374d3275-56a4-4abf-89e3-34e7ee57294c-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:48.064769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.061759    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:48.064769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.061775    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88ftj\" (UniqueName: \"kubernetes.io/projected/374d3275-56a4-4abf-89e3-34e7ee57294c-kube-api-access-88ftj\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:36:48.229367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.229281    5605 generic.go:356] "Generic (PLEG): container finished" podID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerID="c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23" exitCode=0
Nov 03 16:36:48.229367 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.229324    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" event={"ID":"374d3275-56a4-4abf-89e3-34e7ee57294c","Type":"ContainerDied","Data":"c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23"}
Nov 03 16:36:48.229567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.229367    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"
Nov 03 16:36:48.229567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.229403    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm" event={"ID":"374d3275-56a4-4abf-89e3-34e7ee57294c","Type":"ContainerDied","Data":"4b107fdca29f338d53cb304ba6ac0fa2255be15bfc659378dffd455783237574"}
Nov 03 16:36:48.229567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.229436    5605 scope.go:122] "RemoveContainer" containerID="c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23"
Nov 03 16:36:48.231452 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.231414    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd" event={"ID":"1a4e86aa-2c69-4803-994f-99c51afec557","Type":"ContainerStarted","Data":"39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a"}
Nov 03 16:36:48.231622 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.231610    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:48.240392 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.240375    5605 scope.go:122] "RemoveContainer" containerID="9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b"
Nov 03 16:36:48.249636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.249615    5605 scope.go:122] "RemoveContainer" containerID="c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23"
Nov 03 16:36:48.249963 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:36:48.249939    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23\": container with ID starting with c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23 not found: ID does not exist" containerID="c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23"
Nov 03 16:36:48.250035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.249975    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23"} err="failed to get container status \"c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23\": rpc error: code = NotFound desc = could not find container \"c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23\": container with ID starting with c6b61da5a5ea88907d959d787c0dd2052ed89efa9f97769f71f36e7c36e7ad23 not found: ID does not exist"
Nov 03 16:36:48.250035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.250000    5605 scope.go:122] "RemoveContainer" containerID="9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b"
Nov 03 16:36:48.250287 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:36:48.250265    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b\": container with ID starting with 9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b not found: ID does not exist" containerID="9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b"
Nov 03 16:36:48.250346 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.250296    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b"} err="failed to get container status \"9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b\": rpc error: code = NotFound desc = could not find container \"9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b\": container with ID starting with 9605b252760942e7cb76e355c8aaf4c53582bacb7a51136a5f4f644e0878530b not found: ID does not exist"
Nov 03 16:36:48.409057 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.409018    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"]
Nov 03 16:36:48.460876 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.460837    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1b774106-openshift-default-6f9b7f79df-jbwtm"]
Nov 03 16:36:48.583406 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:48.583288    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd" podStartSLOduration=2.695575447 podStartE2EDuration="3.583274439s" totalImagesPullingTime="887.698992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:36:48.582332923 +0000 UTC m=+1494.406784194" watchObservedRunningTime="2026-11-03 16:36:48.583274439 +0000 UTC m=+1494.407725709"
Nov 03 16:36:50.309350 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:50.309318    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" path="/var/lib/kubelet/pods/374d3275-56a4-4abf-89e3-34e7ee57294c/volumes"
Nov 03 16:36:51.359759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.352594    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"]
Nov 03 16:36:51.361193 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.361164    5605 cpu_manager.go:414] "containerMap: removing container" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy"
Nov 03 16:36:51.361350 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.361337    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy"
Nov 03 16:36:51.361490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.361479    5605 cpu_manager.go:414] "containerMap: removing container" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy"
Nov 03 16:36:51.361572 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.361562    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy"
Nov 03 16:36:51.361840 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.361823    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy"
Nov 03 16:36:51.372915 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.372890    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:36:51.397595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.395609    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"]
Nov 03 16:36:51.447692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.442189    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b70a8a53-dockercfg-4rnfr\""
Nov 03 16:36:51.447692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.442283    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b70a8a53\""
Nov 03 16:36:51.463902 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.463863    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnpw9\" (UniqueName: \"kubernetes.io/projected/896b4895-c8a7-4fb8-960d-22a0603a8a34-kube-api-access-mnpw9\") pod \"payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745\" (UID: \"896b4895-c8a7-4fb8-960d-22a0603a8a34\") " pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:36:51.464469 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.464387    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896b4895-c8a7-4fb8-960d-22a0603a8a34-plugins-config-volume\") pod \"payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745\" (UID: \"896b4895-c8a7-4fb8-960d-22a0603a8a34\") " pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:36:51.610478 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.610379    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnpw9\" (UniqueName: \"kubernetes.io/projected/896b4895-c8a7-4fb8-960d-22a0603a8a34-kube-api-access-mnpw9\") pod \"payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745\" (UID: \"896b4895-c8a7-4fb8-960d-22a0603a8a34\") " pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:36:51.610884 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.610861    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896b4895-c8a7-4fb8-960d-22a0603a8a34-plugins-config-volume\") pod \"payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745\" (UID: \"896b4895-c8a7-4fb8-960d-22a0603a8a34\") " pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:36:51.633753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.611903    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896b4895-c8a7-4fb8-960d-22a0603a8a34-plugins-config-volume\") pod \"payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745\" (UID: \"896b4895-c8a7-4fb8-960d-22a0603a8a34\") " pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:36:51.634078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.615240    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"]
Nov 03 16:36:51.638776 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.635430    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="374d3275-56a4-4abf-89e3-34e7ee57294c" containerName="istio-proxy"
Nov 03 16:36:51.790753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.789708    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnpw9\" (UniqueName: \"kubernetes.io/projected/896b4895-c8a7-4fb8-960d-22a0603a8a34-kube-api-access-mnpw9\") pod \"payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745\" (UID: \"896b4895-c8a7-4fb8-960d-22a0603a8a34\") " pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:36:51.811600 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.811574    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:36:51.848753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.821819    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"]
Nov 03 16:36:51.949084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.945932    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z7b4\" (UniqueName: \"kubernetes.io/projected/11f0533a-48e2-49d7-8106-6d1c40baa827-kube-api-access-5z7b4\") pod \"payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4\" (UID: \"11f0533a-48e2-49d7-8106-6d1c40baa827\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:36:51.949084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:51.946192    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f0533a-48e2-49d7-8106-6d1c40baa827-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4\" (UID: \"11f0533a-48e2-49d7-8106-6d1c40baa827\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:36:52.019181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:52.019150    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:36:52.048132 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:52.048094    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z7b4\" (UniqueName: \"kubernetes.io/projected/11f0533a-48e2-49d7-8106-6d1c40baa827-kube-api-access-5z7b4\") pod \"payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4\" (UID: \"11f0533a-48e2-49d7-8106-6d1c40baa827\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:36:52.048366 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:52.048344    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f0533a-48e2-49d7-8106-6d1c40baa827-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4\" (UID: \"11f0533a-48e2-49d7-8106-6d1c40baa827\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:36:52.049404 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:52.049378    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f0533a-48e2-49d7-8106-6d1c40baa827-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4\" (UID: \"11f0533a-48e2-49d7-8106-6d1c40baa827\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:36:52.126219 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:52.124360    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z7b4\" (UniqueName: \"kubernetes.io/projected/11f0533a-48e2-49d7-8106-6d1c40baa827-kube-api-access-5z7b4\") pod \"payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4\" (UID: \"11f0533a-48e2-49d7-8106-6d1c40baa827\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:36:52.219651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:52.219529    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:36:52.409454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:52.409417    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"]
Nov 03 16:36:52.411290 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:36:52.411262    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896b4895_c8a7_4fb8_960d_22a0603a8a34.slice/crio-7e0e00f2f86780acc4c50f079e38e2e85f33dd23837709723ca275dc4ab40a91 WatchSource:0}: Error finding container 7e0e00f2f86780acc4c50f079e38e2e85f33dd23837709723ca275dc4ab40a91: Status 404 returned error can't find the container with id 7e0e00f2f86780acc4c50f079e38e2e85f33dd23837709723ca275dc4ab40a91
Nov 03 16:36:52.612678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:52.612648    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"]
Nov 03 16:36:52.613813 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:36:52.613789    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f0533a_48e2_49d7_8106_6d1c40baa827.slice/crio-a7fa0bde15be8b62ba79b9c6d08a1a7de1c69f4428e0693e0e96419bcdb7e4cf WatchSource:0}: Error finding container a7fa0bde15be8b62ba79b9c6d08a1a7de1c69f4428e0693e0e96419bcdb7e4cf: Status 404 returned error can't find the container with id a7fa0bde15be8b62ba79b9c6d08a1a7de1c69f4428e0693e0e96419bcdb7e4cf
Nov 03 16:36:53.324715 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:53.324674    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4" event={"ID":"11f0533a-48e2-49d7-8106-6d1c40baa827","Type":"ContainerStarted","Data":"c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1"}
Nov 03 16:36:53.324937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:53.324782    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4" event={"ID":"11f0533a-48e2-49d7-8106-6d1c40baa827","Type":"ContainerStarted","Data":"a7fa0bde15be8b62ba79b9c6d08a1a7de1c69f4428e0693e0e96419bcdb7e4cf"}
Nov 03 16:36:53.324937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:53.324832    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:36:53.327116 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:53.327089    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745" event={"ID":"896b4895-c8a7-4fb8-960d-22a0603a8a34","Type":"ContainerStarted","Data":"8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2"}
Nov 03 16:36:53.327227 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:53.327139    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:36:53.327227 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:53.327161    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745" event={"ID":"896b4895-c8a7-4fb8-960d-22a0603a8a34","Type":"ContainerStarted","Data":"7e0e00f2f86780acc4c50f079e38e2e85f33dd23837709723ca275dc4ab40a91"}
Nov 03 16:36:53.469616 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:53.469562    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4" podStartSLOduration=2.469548014 podStartE2EDuration="2.469548014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:36:53.468907951 +0000 UTC m=+1499.293359224" watchObservedRunningTime="2026-11-03 16:36:53.469548014 +0000 UTC m=+1499.293999281"
Nov 03 16:36:53.602635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:53.602571    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745" podStartSLOduration=2.602549632 podStartE2EDuration="2.602549632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:36:53.600167084 +0000 UTC m=+1499.424618378" watchObservedRunningTime="2026-11-03 16:36:53.602549632 +0000 UTC m=+1499.427000904"
Nov 03 16:36:54.244332 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:54.244307    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 16:36:54.504818 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:54.504711    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:36:54.509587 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:54.509562    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:36:54.567629 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:36:54.567599    5605 scope.go:122] "RemoveContainer" containerID="6a41cb2a572d6dd9443532175cd93fd2a296f39676acd51b312b41da9587bf90"
Nov 03 16:37:03.394281 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:37:03.394241    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:37:04.342895 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:04.342868    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:37:04.343100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:04.343018    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:37:04.975355 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:04.975320    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"]
Nov 03 16:37:04.975819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:04.975572    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz" podUID="0f88b986-40aa-4b9d-91e3-083b2a863599" containerName="maas-api" containerID="cri-o://f0d2d57956d8fcdc9989e06c5476bbec709edaa22148b95166b965431b2e3ed1" gracePeriod=30
Nov 03 16:37:05.617524 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.617487    5605 generic.go:356] "Generic (PLEG): container finished" podID="0f88b986-40aa-4b9d-91e3-083b2a863599" containerID="f0d2d57956d8fcdc9989e06c5476bbec709edaa22148b95166b965431b2e3ed1" exitCode=0
Nov 03 16:37:05.618001 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.617976    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz" event={"ID":"0f88b986-40aa-4b9d-91e3-083b2a863599","Type":"ContainerDied","Data":"f0d2d57956d8fcdc9989e06c5476bbec709edaa22148b95166b965431b2e3ed1"}
Nov 03 16:37:05.643115 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.640161    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:37:05.793479 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.793389    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-maas-api-tls\") pod \"0f88b986-40aa-4b9d-91e3-083b2a863599\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") "
Nov 03 16:37:05.793658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.793479    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f88b986-40aa-4b9d-91e3-083b2a863599-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f88b986-40aa-4b9d-91e3-083b2a863599-openshift-service-ca\") pod \"0f88b986-40aa-4b9d-91e3-083b2a863599\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") "
Nov 03 16:37:05.793742 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.793684    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-metrics-certs\") pod \"0f88b986-40aa-4b9d-91e3-083b2a863599\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") "
Nov 03 16:37:05.793806 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.793750    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f88b986-40aa-4b9d-91e3-083b2a863599-kube-api-access-47l56\" (UniqueName: \"kubernetes.io/projected/0f88b986-40aa-4b9d-91e3-083b2a863599-kube-api-access-47l56\") pod \"0f88b986-40aa-4b9d-91e3-083b2a863599\" (UID: \"0f88b986-40aa-4b9d-91e3-083b2a863599\") "
Nov 03 16:37:05.793943 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.793915    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f88b986-40aa-4b9d-91e3-083b2a863599-openshift-service-ca" pod "0f88b986-40aa-4b9d-91e3-083b2a863599" (UID: "0f88b986-40aa-4b9d-91e3-083b2a863599"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:37:05.794597 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.794568    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f88b986-40aa-4b9d-91e3-083b2a863599-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:37:05.796550 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.796522    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-maas-api-tls" pod "0f88b986-40aa-4b9d-91e3-083b2a863599" (UID: "0f88b986-40aa-4b9d-91e3-083b2a863599"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:37:05.796987 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.796960    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-metrics-certs" pod "0f88b986-40aa-4b9d-91e3-083b2a863599" (UID: "0f88b986-40aa-4b9d-91e3-083b2a863599"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:37:05.797125 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.797100    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f88b986-40aa-4b9d-91e3-083b2a863599-kube-api-access-47l56" pod "0f88b986-40aa-4b9d-91e3-083b2a863599" (UID: "0f88b986-40aa-4b9d-91e3-083b2a863599"). InnerVolumeSpecName "kube-api-access-47l56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:37:05.896210 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.896174    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:37:05.896210 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.896209    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f88b986-40aa-4b9d-91e3-083b2a863599-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:37:05.896484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:05.896224    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47l56\" (UniqueName: \"kubernetes.io/projected/0f88b986-40aa-4b9d-91e3-083b2a863599-kube-api-access-47l56\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:37:06.626425 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:06.626396    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"
Nov 03 16:37:06.626933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:06.626433    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz" event={"ID":"0f88b986-40aa-4b9d-91e3-083b2a863599","Type":"ContainerDied","Data":"872f1aab3c1f7e19de607bc404ecec98abbb17b7861ea08bc4178619243d4d7d"}
Nov 03 16:37:06.626933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:06.626500    5605 scope.go:122] "RemoveContainer" containerID="f0d2d57956d8fcdc9989e06c5476bbec709edaa22148b95166b965431b2e3ed1"
Nov 03 16:37:06.804111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:06.804081    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"]
Nov 03 16:37:06.810359 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:06.810322    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e6def865-75b8b66b85-m65nz"]
Nov 03 16:37:08.309452 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.309418    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f88b986-40aa-4b9d-91e3-083b2a863599" path="/var/lib/kubelet/pods/0f88b986-40aa-4b9d-91e3-083b2a863599/volumes"
Nov 03 16:37:08.886558 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.886525    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"]
Nov 03 16:37:08.887165 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.887150    5605 cpu_manager.go:414] "containerMap: removing container" podUID="0f88b986-40aa-4b9d-91e3-083b2a863599" containerName="maas-api"
Nov 03 16:37:08.887220 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.887166    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f88b986-40aa-4b9d-91e3-083b2a863599" containerName="maas-api"
Nov 03 16:37:08.887296 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.887277    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f88b986-40aa-4b9d-91e3-083b2a863599" containerName="maas-api"
Nov 03 16:37:08.890870 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.890852    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:08.954189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.954156    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:08.954361 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.954314    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:08.954416 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.954399    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:08.954453 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.954447    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:08.954499 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.954487    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:08.954572 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.954553    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:08.954606 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.954585    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:08.954640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.954607    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03b5a7fa-c4c4-4778-a652-09969f86569e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:08.954640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.954628    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcd5x\" (UniqueName: \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-kube-api-access-gcd5x\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:08.996117 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:08.996082    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"]
Nov 03 16:37:09.056609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.056565    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.056813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.056675    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.056813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.056749    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.056813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.056791    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.057019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.056832    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.057019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.056851    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.057019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.056870    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.057019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.056888    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03b5a7fa-c4c4-4778-a652-09969f86569e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.057019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.056910    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcd5x\" (UniqueName: \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-kube-api-access-gcd5x\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.057279 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.057092    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.057342 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.057316    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.057550 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.057516    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.057842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.057818    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.058039 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.058011    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03b5a7fa-c4c4-4778-a652-09969f86569e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.060060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.060029    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.060167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.060055    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.119188 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.119159    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcd5x\" (UniqueName: \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-kube-api-access-gcd5x\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.119378 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.119209    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-vlxdg\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.204330 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.204244    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:09.652998 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.652970    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"]
Nov 03 16:37:09.653610 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:37:09.653581    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b5a7fa_c4c4_4778_a652_09969f86569e.slice/crio-ef7194693f5746b874aff4893c5623b502587aca92a922a5e2e96bd65f1f66b8 WatchSource:0}: Error finding container ef7194693f5746b874aff4893c5623b502587aca92a922a5e2e96bd65f1f66b8: Status 404 returned error can't find the container with id ef7194693f5746b874aff4893c5623b502587aca92a922a5e2e96bd65f1f66b8
Nov 03 16:37:09.656289 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.656256    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:37:09.656349 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.656327    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:37:09.656384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:09.656357    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:37:10.661571 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:10.661537    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" event={"ID":"03b5a7fa-c4c4-4778-a652-09969f86569e","Type":"ContainerStarted","Data":"3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e"}
Nov 03 16:37:10.662070 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:10.661589    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" event={"ID":"03b5a7fa-c4c4-4778-a652-09969f86569e","Type":"ContainerStarted","Data":"ef7194693f5746b874aff4893c5623b502587aca92a922a5e2e96bd65f1f66b8"}
Nov 03 16:37:10.841820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:10.841767    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" podStartSLOduration=2.841716393 podStartE2EDuration="2.841716393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:37:10.840422514 +0000 UTC m=+1516.664873784" watchObservedRunningTime="2026-11-03 16:37:10.841716393 +0000 UTC m=+1516.666167663"
Nov 03 16:37:11.205915 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:11.205880    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:11.208150 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:11.208119    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-vlxdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 03 16:37:11.208308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:11.208177    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 03 16:37:12.245075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:12.240869    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-vlxdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 03 16:37:12.245075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:12.240951    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 03 16:37:13.206899 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:13.205971    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-vlxdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 03 16:37:13.206899 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:13.206032    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 03 16:37:14.206750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:14.206056    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-vlxdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 03 16:37:14.206750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:14.206116    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 03 16:37:15.206135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:15.206094    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-vlxdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 03 16:37:15.206340 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:15.206160    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 03 16:37:16.206096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:16.206059    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-vlxdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 03 16:37:16.206540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:16.206124    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 03 16:37:17.205050 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:17.205012    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-vlxdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 03 16:37:17.205230 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:17.205079    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 03 16:37:18.787755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:18.787697    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:18.788373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:18.788349    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:18.789350 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:18.789330    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:37:23.878456 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.878423    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"]
Nov 03 16:37:23.883883 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.883864    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:23.967285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.967249    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:23.967440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.967295    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:23.967440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.967370    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/266192f8-6a4e-4736-b400-22433bd0e632-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:23.967440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.967423    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/266192f8-6a4e-4736-b400-22433bd0e632-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:23.967606 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.967459    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:23.967663 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.967626    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:23.967719 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.967663    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:23.967807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.967765    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cfd5\" (UniqueName: \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-kube-api-access-6cfd5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:23.967862 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.967832    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:23.995544 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:23.995504    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"]
Nov 03 16:37:24.069674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.069637    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.069674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.069675    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.069967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.069703    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cfd5\" (UniqueName: \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-kube-api-access-6cfd5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.069967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.069760    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.069967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.069906    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.069967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.069943    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.070183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.069998    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/266192f8-6a4e-4736-b400-22433bd0e632-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.070183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.070026    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/266192f8-6a4e-4736-b400-22433bd0e632-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.070183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.070070    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.070183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.070076    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.070183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.070133    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.070388 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.070192    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.070388 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.070307    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.070621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.070603    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/266192f8-6a4e-4736-b400-22433bd0e632-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.072385 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.072369    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.072446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.072427    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/266192f8-6a4e-4736-b400-22433bd0e632-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.132339 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.132274    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.132462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.132382    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cfd5\" (UniqueName: \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-kube-api-access-6cfd5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-kklqk\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.198158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.198131    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:24.637425 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.637395    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"]
Nov 03 16:37:24.637845 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:37:24.637813    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266192f8_6a4e_4736_b400_22433bd0e632.slice/crio-78908e3abc2edfeb173678ce6c0f9b691b91d6b7e0a1e5320bada9bc48370d90 WatchSource:0}: Error finding container 78908e3abc2edfeb173678ce6c0f9b691b91d6b7e0a1e5320bada9bc48370d90: Status 404 returned error can't find the container with id 78908e3abc2edfeb173678ce6c0f9b691b91d6b7e0a1e5320bada9bc48370d90
Nov 03 16:37:24.640350 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.640317    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:37:24.640424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.640385    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:37:24.640424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.640415    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:37:24.802868 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.802830    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" event={"ID":"266192f8-6a4e-4736-b400-22433bd0e632","Type":"ContainerStarted","Data":"086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc"}
Nov 03 16:37:24.803083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.802893    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" event={"ID":"266192f8-6a4e-4736-b400-22433bd0e632","Type":"ContainerStarted","Data":"78908e3abc2edfeb173678ce6c0f9b691b91d6b7e0a1e5320bada9bc48370d90"}
Nov 03 16:37:24.927624 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:24.927514    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" podStartSLOduration=1.927494891 podStartE2EDuration="1.927494891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:37:24.926864765 +0000 UTC m=+1530.751316050" watchObservedRunningTime="2026-11-03 16:37:24.927494891 +0000 UTC m=+1530.751946161"
Nov 03 16:37:25.206834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:25.205902    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:25.207017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:25.206994    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-kklqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 16:37:25.207117 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:25.207045    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 16:37:26.206224 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:26.206185    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-kklqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 16:37:26.206905 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:26.206247    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 16:37:27.207654 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:27.207164    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-kklqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 16:37:27.207654 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:27.207226    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 16:37:28.200999 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:28.200963    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-kklqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 16:37:28.201289 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:28.201257    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 16:37:29.202129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:29.202087    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-kklqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 16:37:29.202777 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:29.202742    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 16:37:30.207192 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:30.207148    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-kklqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 16:37:30.207656 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:30.207213    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 16:37:31.199433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:31.199398    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-kklqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 16:37:31.199613 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:31.199458    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 16:37:32.199283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:32.199250    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-kklqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 16:37:32.199694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:32.199313    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 16:37:33.395811 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:37:33.395770    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:37:33.463747 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:33.463691    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:33.464303 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:33.464272    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:33.465087 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:33.465063    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:37:39.344714 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.344677    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"]
Nov 03 16:37:39.360940 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.360915    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.477755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.476064    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"]
Nov 03 16:37:39.550749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.546293    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b70a8a53-serving-cert\""
Nov 03 16:37:39.568377 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.568279    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ae65c7-7bbb-49ac-8876-db9f7d84031c-openshift-service-ca\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.568684 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.568663    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-metrics-certs\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.568933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.568911    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq7xj\" (UniqueName: \"kubernetes.io/projected/59ae65c7-7bbb-49ac-8876-db9f7d84031c-kube-api-access-rq7xj\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.569124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.569108    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-maas-api-tls\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.803832 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.785068    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ae65c7-7bbb-49ac-8876-db9f7d84031c-openshift-service-ca\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.803832 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.785147    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-metrics-certs\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.803832 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.785177    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq7xj\" (UniqueName: \"kubernetes.io/projected/59ae65c7-7bbb-49ac-8876-db9f7d84031c-kube-api-access-rq7xj\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.803832 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.785211    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-maas-api-tls\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.821762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.812550    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ae65c7-7bbb-49ac-8876-db9f7d84031c-openshift-service-ca\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.863822 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.863751    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-metrics-certs\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.934474 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.934434    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-maas-api-tls\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:39.937943 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:39.937877    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq7xj\" (UniqueName: \"kubernetes.io/projected/59ae65c7-7bbb-49ac-8876-db9f7d84031c-kube-api-access-rq7xj\") pod \"maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:40.018814 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:40.015041    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:41.054822 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:37:41.051022    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ae65c7_7bbb_49ac_8876_db9f7d84031c.slice/crio-1fafeb71a4d6d139e7d8dc80ed862d7ebef60c9ce104f4aaec45c6ce92e23829 WatchSource:0}: Error finding container 1fafeb71a4d6d139e7d8dc80ed862d7ebef60c9ce104f4aaec45c6ce92e23829: Status 404 returned error can't find the container with id 1fafeb71a4d6d139e7d8dc80ed862d7ebef60c9ce104f4aaec45c6ce92e23829
Nov 03 16:37:41.078760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:41.077138    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"]
Nov 03 16:37:42.155483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:42.155421    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9" event={"ID":"59ae65c7-7bbb-49ac-8876-db9f7d84031c","Type":"ContainerStarted","Data":"1fafeb71a4d6d139e7d8dc80ed862d7ebef60c9ce104f4aaec45c6ce92e23829"}
Nov 03 16:37:43.163743 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:43.163688    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9" event={"ID":"59ae65c7-7bbb-49ac-8876-db9f7d84031c","Type":"ContainerStarted","Data":"4d0d3a03f9d06a3142b56ea513c47580f632b3c92cbf471952a80eeaf4f6fc7f"}
Nov 03 16:37:43.164182 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:43.163793    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:43.289202 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:43.289151    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9" podStartSLOduration=3.293075426 podStartE2EDuration="4.289136513s" totalImagesPullingTime="996.061087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:37:43.287326389 +0000 UTC m=+1549.111777662" watchObservedRunningTime="2026-11-03 16:37:43.289136513 +0000 UTC m=+1549.113587781"
Nov 03 16:37:49.160754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.153635    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"]
Nov 03 16:37:49.171757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.170958    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:37:49.202109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.193962    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:37:49.244127 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.243404    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpns6\" (UniqueName: \"kubernetes.io/projected/7ce2633c-8018-433b-8afc-7928e26a560b-kube-api-access-rpns6\") pod \"payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg\" (UID: \"7ce2633c-8018-433b-8afc-7928e26a560b\") " pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:37:49.244127 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.243465    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce2633c-8018-433b-8afc-7928e26a560b-plugins-config-volume\") pod \"payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg\" (UID: \"7ce2633c-8018-433b-8afc-7928e26a560b\") " pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:37:49.246278 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.246247    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"]
Nov 03 16:37:49.323892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.307154    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9635db0d\""
Nov 03 16:37:49.323892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.307329    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9635db0d-dockercfg-gtsxf\""
Nov 03 16:37:49.475912 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.475823    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpns6\" (UniqueName: \"kubernetes.io/projected/7ce2633c-8018-433b-8afc-7928e26a560b-kube-api-access-rpns6\") pod \"payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg\" (UID: \"7ce2633c-8018-433b-8afc-7928e26a560b\") " pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:37:49.476170 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.476148    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce2633c-8018-433b-8afc-7928e26a560b-plugins-config-volume\") pod \"payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg\" (UID: \"7ce2633c-8018-433b-8afc-7928e26a560b\") " pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:37:49.485591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.479549    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce2633c-8018-433b-8afc-7928e26a560b-plugins-config-volume\") pod \"payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg\" (UID: \"7ce2633c-8018-433b-8afc-7928e26a560b\") " pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:37:49.747988 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.747050    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpns6\" (UniqueName: \"kubernetes.io/projected/7ce2633c-8018-433b-8afc-7928e26a560b-kube-api-access-rpns6\") pod \"payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg\" (UID: \"7ce2633c-8018-433b-8afc-7928e26a560b\") " pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:37:49.879754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:49.805656    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:37:50.464152 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:50.462380    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"]
Nov 03 16:37:50.761929 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:50.761846    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg" event={"ID":"7ce2633c-8018-433b-8afc-7928e26a560b","Type":"ContainerStarted","Data":"96dc0b5f0d8524e969a4746543c4a68b77db0555e1a1952a204cedcd14163c7c"}
Nov 03 16:37:50.761929 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:50.761895    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg" event={"ID":"7ce2633c-8018-433b-8afc-7928e26a560b","Type":"ContainerStarted","Data":"c0c1a6ab4ca7cf4d1b500b5644b42d0ab7f5d69b7656f5eba385e176e888fa33"}
Nov 03 16:37:50.762121 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:50.761971    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:37:50.884961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:50.884910    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg" podStartSLOduration=1.8848951139999999 podStartE2EDuration="1.884895114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:37:50.882187435 +0000 UTC m=+1556.706638706" watchObservedRunningTime="2026-11-03 16:37:50.884895114 +0000 UTC m=+1556.709346385"
Nov 03 16:37:53.983505 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:53.975758    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"]
Nov 03 16:37:53.987751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:53.986827    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.009160 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.009113    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.009348 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.009193    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.009348 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.009261    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.009348 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.009283    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6e016ec-3eaf-4d13-a322-425d83281dc0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.009348 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.009332    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.009562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.009364    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.009562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.009474    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.023786 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.009700    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxpgb\" (UniqueName: \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-kube-api-access-jxpgb\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.023786 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.009788    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.094238 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.078830    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"]
Nov 03 16:37:54.127521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.127348    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.127521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.127396    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6e016ec-3eaf-4d13-a322-425d83281dc0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.127521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.127453    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.127521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.127496    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.128004 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.127592    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.128064 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.127997    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxpgb\" (UniqueName: \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-kube-api-access-jxpgb\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.128123 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.128092    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.128787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.128189    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.128787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.128250    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.128787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.128767    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6e016ec-3eaf-4d13-a322-425d83281dc0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.129010 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.128887    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.129745 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.129090    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.129745 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.129223    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.129745 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.129480    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.299908 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.299834    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.312720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.305845    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxpgb\" (UniqueName: \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-kube-api-access-jxpgb\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.326834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.326801    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.394771 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.384890    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-trzd6\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:54.662954 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:54.662923    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:55.302984 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:55.302952    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"]
Nov 03 16:37:55.339938 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:37:55.339891    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e016ec_3eaf_4d13_a322_425d83281dc0.slice/crio-59b53b0a9b09b4bd67aa3fbd68c76f56fe389077778ed8be2b9677bad6c60df5 WatchSource:0}: Error finding container 59b53b0a9b09b4bd67aa3fbd68c76f56fe389077778ed8be2b9677bad6c60df5: Status 404 returned error can't find the container with id 59b53b0a9b09b4bd67aa3fbd68c76f56fe389077778ed8be2b9677bad6c60df5
Nov 03 16:37:55.343662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:55.343616    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:37:55.343855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:55.343714    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:37:55.343855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:55.343783    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:37:56.125913 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:56.124575    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" event={"ID":"e6e016ec-3eaf-4d13-a322-425d83281dc0","Type":"ContainerStarted","Data":"0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4"}
Nov 03 16:37:56.125913 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:56.124644    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" event={"ID":"e6e016ec-3eaf-4d13-a322-425d83281dc0","Type":"ContainerStarted","Data":"59b53b0a9b09b4bd67aa3fbd68c76f56fe389077778ed8be2b9677bad6c60df5"}
Nov 03 16:37:56.316641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:56.316584    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" podStartSLOduration=3.316568522 podStartE2EDuration="3.316568522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:37:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:37:56.303963434 +0000 UTC m=+1562.128414706" watchObservedRunningTime="2026-11-03 16:37:56.316568522 +0000 UTC m=+1562.141019793"
Nov 03 16:37:56.663519 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:56.663487    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:37:56.664819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:56.664798    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-trzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 03 16:37:56.664895 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:56.664842    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 03 16:37:57.663594 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:57.663559    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-trzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 03 16:37:57.664017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:57.663607    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 03 16:37:58.663945 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:58.663849    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-trzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 03 16:37:58.663945 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:58.663910    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 03 16:37:59.664048 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:59.664018    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-trzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 03 16:37:59.664438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:59.664083    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 03 16:37:59.954893 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:59.954801    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"]
Nov 03 16:37:59.955055 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:37:59.955003    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2" podUID="dffeb6e9-03ca-4250-9032-bbe295bbf959" containerName="maas-api" containerID="cri-o://aed836f0b869ff6dfb65e9f57ced813275d9e25032d2843c5a5bf2b30a639158" gracePeriod=30
Nov 03 16:38:00.155081 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:00.155039    5605 generic.go:356] "Generic (PLEG): container finished" podID="dffeb6e9-03ca-4250-9032-bbe295bbf959" containerID="aed836f0b869ff6dfb65e9f57ced813275d9e25032d2843c5a5bf2b30a639158" exitCode=0
Nov 03 16:38:00.155237 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:00.155090    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2" event={"ID":"dffeb6e9-03ca-4250-9032-bbe295bbf959","Type":"ContainerDied","Data":"aed836f0b869ff6dfb65e9f57ced813275d9e25032d2843c5a5bf2b30a639158"}
Nov 03 16:38:00.909086 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:00.909043    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-trzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 03 16:38:00.909744 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:00.909684    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 03 16:38:00.911356 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:00.911330    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:38:01.112754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.103042    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dffeb6e9-03ca-4250-9032-bbe295bbf959-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dffeb6e9-03ca-4250-9032-bbe295bbf959-openshift-service-ca\") pod \"dffeb6e9-03ca-4250-9032-bbe295bbf959\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") "
Nov 03 16:38:01.112754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.103319    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-maas-api-tls\") pod \"dffeb6e9-03ca-4250-9032-bbe295bbf959\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") "
Nov 03 16:38:01.112754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.103359    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-metrics-certs\") pod \"dffeb6e9-03ca-4250-9032-bbe295bbf959\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") "
Nov 03 16:38:01.112754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.103493    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dffeb6e9-03ca-4250-9032-bbe295bbf959-kube-api-access-ts86m\" (UniqueName: \"kubernetes.io/projected/dffeb6e9-03ca-4250-9032-bbe295bbf959-kube-api-access-ts86m\") pod \"dffeb6e9-03ca-4250-9032-bbe295bbf959\" (UID: \"dffeb6e9-03ca-4250-9032-bbe295bbf959\") "
Nov 03 16:38:01.112754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.105135    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dffeb6e9-03ca-4250-9032-bbe295bbf959-openshift-service-ca" pod "dffeb6e9-03ca-4250-9032-bbe295bbf959" (UID: "dffeb6e9-03ca-4250-9032-bbe295bbf959"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:38:01.201143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.182915    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dffeb6e9-03ca-4250-9032-bbe295bbf959-kube-api-access-ts86m" pod "dffeb6e9-03ca-4250-9032-bbe295bbf959" (UID: "dffeb6e9-03ca-4250-9032-bbe295bbf959"). InnerVolumeSpecName "kube-api-access-ts86m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:38:01.201143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.183007    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-metrics-certs" pod "dffeb6e9-03ca-4250-9032-bbe295bbf959" (UID: "dffeb6e9-03ca-4250-9032-bbe295bbf959"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:38:01.201143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.193800    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-maas-api-tls" pod "dffeb6e9-03ca-4250-9032-bbe295bbf959" (UID: "dffeb6e9-03ca-4250-9032-bbe295bbf959"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:38:01.255792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.244579    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:01.256105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.256080    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dffeb6e9-03ca-4250-9032-bbe295bbf959-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:01.256294 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.256272    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts86m\" (UniqueName: \"kubernetes.io/projected/dffeb6e9-03ca-4250-9032-bbe295bbf959-kube-api-access-ts86m\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:01.256407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.256392    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dffeb6e9-03ca-4250-9032-bbe295bbf959-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:01.403785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.386813    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2" event={"ID":"dffeb6e9-03ca-4250-9032-bbe295bbf959","Type":"ContainerDied","Data":"9dc0a3c9bcab013ada1c3388c9c2fe15ecf5082a82a55a960161d2b761fd775c"}
Nov 03 16:38:01.404157 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.404126    5605 scope.go:122] "RemoveContainer" containerID="aed836f0b869ff6dfb65e9f57ced813275d9e25032d2843c5a5bf2b30a639158"
Nov 03 16:38:01.404525 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.404504    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"
Nov 03 16:38:01.658668 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.658637    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"]
Nov 03 16:38:01.663323 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.663295    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-trzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 03 16:38:01.663477 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.663368    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 03 16:38:01.716134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.716089    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06f127ec-b4d774b58-wgpj2"]
Nov 03 16:38:01.772164 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:01.772084    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:38:02.418594 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:02.418556    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dffeb6e9-03ca-4250-9032-bbe295bbf959" path="/var/lib/kubelet/pods/dffeb6e9-03ca-4250-9032-bbe295bbf959/volumes"
Nov 03 16:38:02.664194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:02.664151    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-trzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 03 16:38:02.664370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:02.664228    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 03 16:38:03.395495 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:38:03.395457    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:38:03.663642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:03.663556    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-trzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 03 16:38:03.663642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:03.663629    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 03 16:38:03.882270 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:03.882226    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"]
Nov 03 16:38:03.882574 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:03.882545    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy" containerID="cri-o://bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1" gracePeriod=30
Nov 03 16:38:03.931299 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:03.931229    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"]
Nov 03 16:38:03.931471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:03.931431    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerName="istio-proxy" containerID="cri-o://aef0d03426c583f3bc4bac6ddf427a0b8e147891de38b3b8eec54f93e5b4561d" gracePeriod=30
Nov 03 16:38:04.778702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:04.778670    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:38:04.779189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:04.779168    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:38:04.780109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:04.780080    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:38:06.754522 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.753523    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"]
Nov 03 16:38:06.754522 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.754512    5605 cpu_manager.go:414] "containerMap: removing container" podUID="dffeb6e9-03ca-4250-9032-bbe295bbf959" containerName="maas-api"
Nov 03 16:38:06.755097 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.754532    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dffeb6e9-03ca-4250-9032-bbe295bbf959" containerName="maas-api"
Nov 03 16:38:06.755097 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.754793    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="dffeb6e9-03ca-4250-9032-bbe295bbf959" containerName="maas-api"
Nov 03 16:38:06.761561 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.761530    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 16:38:06.812354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.810477    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"]
Nov 03 16:38:06.847425 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.847395    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-509f14\""
Nov 03 16:38:06.847839 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.847819    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-509f14-dockercfg-m47z7\""
Nov 03 16:38:06.870658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.868953    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889\" (UID: \"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 16:38:06.870658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.869398    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqfk2\" (UniqueName: \"kubernetes.io/projected/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-kube-api-access-wqfk2\") pod \"payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889\" (UID: \"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 16:38:06.977750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.976911    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqfk2\" (UniqueName: \"kubernetes.io/projected/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-kube-api-access-wqfk2\") pod \"payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889\" (UID: \"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 16:38:06.977750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.977054    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889\" (UID: \"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 16:38:06.985161 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.978919    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"]
Nov 03 16:38:06.995854 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:06.986783    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889\" (UID: \"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 16:38:07.068912 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.066415    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 16:38:07.122772 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.121077    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqfk2\" (UniqueName: \"kubernetes.io/projected/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-kube-api-access-wqfk2\") pod \"payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889\" (UID: \"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 16:38:07.155395 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.154661    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"]
Nov 03 16:38:07.192585 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.192542    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/375c1022-4008-4245-8007-54fede100898-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g\" (UID: \"375c1022-4008-4245-8007-54fede100898\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 16:38:07.192961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.192937    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn4lc\" (UniqueName: \"kubernetes.io/projected/375c1022-4008-4245-8007-54fede100898-kube-api-access-sn4lc\") pod \"payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g\" (UID: \"375c1022-4008-4245-8007-54fede100898\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 16:38:07.327193 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.327108    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/375c1022-4008-4245-8007-54fede100898-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g\" (UID: \"375c1022-4008-4245-8007-54fede100898\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 16:38:07.327537 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.327518    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn4lc\" (UniqueName: \"kubernetes.io/projected/375c1022-4008-4245-8007-54fede100898-kube-api-access-sn4lc\") pod \"payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g\" (UID: \"375c1022-4008-4245-8007-54fede100898\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 16:38:07.329497 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.329470    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/375c1022-4008-4245-8007-54fede100898-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g\" (UID: \"375c1022-4008-4245-8007-54fede100898\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 16:38:07.411757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.410202    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 16:38:07.411757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.411395    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn4lc\" (UniqueName: \"kubernetes.io/projected/375c1022-4008-4245-8007-54fede100898-kube-api-access-sn4lc\") pod \"payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g\" (UID: \"375c1022-4008-4245-8007-54fede100898\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 16:38:07.472995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:07.471085    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 16:38:08.048767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:08.047826    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"]
Nov 03 16:38:08.049420 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:38:08.049392    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d769bc_b56e_4e04_9d8b_157df8b7e6c2.slice/crio-1a84a959e7b1895642d2dcf5301b18ee32f7d199f628dd8f406b2a9a2ce6280e WatchSource:0}: Error finding container 1a84a959e7b1895642d2dcf5301b18ee32f7d199f628dd8f406b2a9a2ce6280e: Status 404 returned error can't find the container with id 1a84a959e7b1895642d2dcf5301b18ee32f7d199f628dd8f406b2a9a2ce6280e
Nov 03 16:38:08.099684 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:08.099658    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"]
Nov 03 16:38:08.109017 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:38:08.108989    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375c1022_4008_4245_8007_54fede100898.slice/crio-2a17cdf050c3e74757ae66367fbdbb99c59db328f6962552dec1d8d8a0ce1064 WatchSource:0}: Error finding container 2a17cdf050c3e74757ae66367fbdbb99c59db328f6962552dec1d8d8a0ce1064: Status 404 returned error can't find the container with id 2a17cdf050c3e74757ae66367fbdbb99c59db328f6962552dec1d8d8a0ce1064
Nov 03 16:38:08.691966 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:08.691933    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889" event={"ID":"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2","Type":"ContainerStarted","Data":"abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510"}
Nov 03 16:38:08.692155 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:08.691995    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889" event={"ID":"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2","Type":"ContainerStarted","Data":"1a84a959e7b1895642d2dcf5301b18ee32f7d199f628dd8f406b2a9a2ce6280e"}
Nov 03 16:38:08.692155 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:08.692024    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 16:38:08.693635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:08.693610    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g" event={"ID":"375c1022-4008-4245-8007-54fede100898","Type":"ContainerStarted","Data":"faa45ac442e7c42390e666c1c5265bf5b7804d37c105cafadf7885bd862b8e47"}
Nov 03 16:38:08.694256 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:08.693653    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g" event={"ID":"375c1022-4008-4245-8007-54fede100898","Type":"ContainerStarted","Data":"2a17cdf050c3e74757ae66367fbdbb99c59db328f6962552dec1d8d8a0ce1064"}
Nov 03 16:38:08.694256 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:08.693705    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 16:38:08.821276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:08.821220    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889" podStartSLOduration=2.8212061139999998 podStartE2EDuration="2.821206114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:38:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:38:08.818910528 +0000 UTC m=+1574.643361799" watchObservedRunningTime="2026-11-03 16:38:08.821206114 +0000 UTC m=+1574.645657385"
Nov 03 16:38:08.945064 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:08.944969    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g" podStartSLOduration=2.944954926 podStartE2EDuration="2.944954926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:38:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:38:08.94088074 +0000 UTC m=+1574.765332015" watchObservedRunningTime="2026-11-03 16:38:08.944954926 +0000 UTC m=+1574.769406196"
Nov 03 16:38:09.674212 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.674177    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:38:09.706341 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.706313    5605 generic.go:356] "Generic (PLEG): container finished" podID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerID="bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1" exitCode=0
Nov 03 16:38:09.706532 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.706520    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"
Nov 03 16:38:09.706662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.706647    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" event={"ID":"f6d14b41-06d9-4fce-92fd-de9a147d3316","Type":"ContainerDied","Data":"bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1"}
Nov 03 16:38:09.706714 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.706683    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" event={"ID":"f6d14b41-06d9-4fce-92fd-de9a147d3316","Type":"ContainerDied","Data":"5b1d794d4f4841d5404fb9a20a177d603f8017f621baa45630ce369792da1eb1"}
Nov 03 16:38:09.706714 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.706707    5605 scope.go:122] "RemoveContainer" containerID="bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1"
Nov 03 16:38:09.709362 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.709337    5605 generic.go:356] "Generic (PLEG): container finished" podID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerID="aef0d03426c583f3bc4bac6ddf427a0b8e147891de38b3b8eec54f93e5b4561d" exitCode=0
Nov 03 16:38:09.709474 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.709454    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" event={"ID":"4e1d654b-6a17-473c-bcb4-5933e36769c5","Type":"ContainerDied","Data":"aef0d03426c583f3bc4bac6ddf427a0b8e147891de38b3b8eec54f93e5b4561d"}
Nov 03 16:38:09.728857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.728707    5605 scope.go:122] "RemoveContainer" containerID="bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1"
Nov 03 16:38:09.729325 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:38:09.729301    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1\": container with ID starting with bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1 not found: ID does not exist" containerID="bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1"
Nov 03 16:38:09.729483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.729461    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1"} err="failed to get container status \"bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1\": rpc error: code = NotFound desc = could not find container \"bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1\": container with ID starting with bdb8fb5f020939a469d08d9646bc92c59c390ce9d7beae3a1194e3daf2435dd1 not found: ID does not exist"
Nov 03 16:38:09.759620 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.759583    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-socket\") pod \"f6d14b41-06d9-4fce-92fd-de9a147d3316\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") "
Nov 03 16:38:09.759718 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.759642    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-data\") pod \"f6d14b41-06d9-4fce-92fd-de9a147d3316\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") "
Nov 03 16:38:09.759718 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.759694    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-envoy\") pod \"f6d14b41-06d9-4fce-92fd-de9a147d3316\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") "
Nov 03 16:38:09.759840 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.759800    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-credential-socket\") pod \"f6d14b41-06d9-4fce-92fd-de9a147d3316\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") "
Nov 03 16:38:09.759892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.759859    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-socket" pod "f6d14b41-06d9-4fce-92fd-de9a147d3316" (UID: "f6d14b41-06d9-4fce-92fd-de9a147d3316"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:38:09.759892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.759867    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-data" pod "f6d14b41-06d9-4fce-92fd-de9a147d3316" (UID: "f6d14b41-06d9-4fce-92fd-de9a147d3316"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:38:09.759892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.759872    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6d14b41-06d9-4fce-92fd-de9a147d3316-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6d14b41-06d9-4fce-92fd-de9a147d3316-istiod-ca-cert\") pod \"f6d14b41-06d9-4fce-92fd-de9a147d3316\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") "
Nov 03 16:38:09.759993 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.759979    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-token\" (UniqueName: \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-token\") pod \"f6d14b41-06d9-4fce-92fd-de9a147d3316\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") "
Nov 03 16:38:09.760117 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.760091    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-credential-socket" pod "f6d14b41-06d9-4fce-92fd-de9a147d3316" (UID: "f6d14b41-06d9-4fce-92fd-de9a147d3316"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:38:09.760232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.760152    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-podinfo\") pod \"f6d14b41-06d9-4fce-92fd-de9a147d3316\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") "
Nov 03 16:38:09.760232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.760167    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6d14b41-06d9-4fce-92fd-de9a147d3316-istiod-ca-cert" pod "f6d14b41-06d9-4fce-92fd-de9a147d3316" (UID: "f6d14b41-06d9-4fce-92fd-de9a147d3316"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:38:09.760232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.760193    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-kube-api-access-dj8cp\" (UniqueName: \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-kube-api-access-dj8cp\") pod \"f6d14b41-06d9-4fce-92fd-de9a147d3316\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") "
Nov 03 16:38:09.760393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.760338    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-certs\") pod \"f6d14b41-06d9-4fce-92fd-de9a147d3316\" (UID: \"f6d14b41-06d9-4fce-92fd-de9a147d3316\") "
Nov 03 16:38:09.760658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.760635    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-certs" pod "f6d14b41-06d9-4fce-92fd-de9a147d3316" (UID: "f6d14b41-06d9-4fce-92fd-de9a147d3316"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:38:09.761149 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.761132    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.761255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.761154    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.761255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.761168    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.761255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.761182    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.761255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.761194    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6d14b41-06d9-4fce-92fd-de9a147d3316-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.762478 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.762457    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-kube-api-access-dj8cp" pod "f6d14b41-06d9-4fce-92fd-de9a147d3316" (UID: "f6d14b41-06d9-4fce-92fd-de9a147d3316"). InnerVolumeSpecName "kube-api-access-dj8cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:38:09.762582 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.762549    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-envoy" pod "f6d14b41-06d9-4fce-92fd-de9a147d3316" (UID: "f6d14b41-06d9-4fce-92fd-de9a147d3316"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:38:09.762819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.762801    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-token" pod "f6d14b41-06d9-4fce-92fd-de9a147d3316" (UID: "f6d14b41-06d9-4fce-92fd-de9a147d3316"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:38:09.763736 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.763702    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-podinfo" pod "f6d14b41-06d9-4fce-92fd-de9a147d3316" (UID: "f6d14b41-06d9-4fce-92fd-de9a147d3316"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:38:09.794823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.794804    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:38:09.861773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.861740    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-certs\") pod \"4e1d654b-6a17-473c-bcb4-5933e36769c5\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") "
Nov 03 16:38:09.861949 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.861865    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-token\" (UniqueName: \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-token\") pod \"4e1d654b-6a17-473c-bcb4-5933e36769c5\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") "
Nov 03 16:38:09.862013 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.861985    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-certs" pod "4e1d654b-6a17-473c-bcb4-5933e36769c5" (UID: "4e1d654b-6a17-473c-bcb4-5933e36769c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:38:09.862013 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.861995    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-socket\") pod \"4e1d654b-6a17-473c-bcb4-5933e36769c5\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") "
Nov 03 16:38:09.862118 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.862041    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e1d654b-6a17-473c-bcb4-5933e36769c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e1d654b-6a17-473c-bcb4-5933e36769c5-istiod-ca-cert\") pod \"4e1d654b-6a17-473c-bcb4-5933e36769c5\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") "
Nov 03 16:38:09.862118 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.862078    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-podinfo\") pod \"4e1d654b-6a17-473c-bcb4-5933e36769c5\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") "
Nov 03 16:38:09.862252 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.862230    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-data\") pod \"4e1d654b-6a17-473c-bcb4-5933e36769c5\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") "
Nov 03 16:38:09.862411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.862276    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-envoy\") pod \"4e1d654b-6a17-473c-bcb4-5933e36769c5\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") "
Nov 03 16:38:09.862411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.862381    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-kube-api-access-5nzxf\" (UniqueName: \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-kube-api-access-5nzxf\") pod \"4e1d654b-6a17-473c-bcb4-5933e36769c5\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") "
Nov 03 16:38:09.862587 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.862432    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-credential-socket\") pod \"4e1d654b-6a17-473c-bcb4-5933e36769c5\" (UID: \"4e1d654b-6a17-473c-bcb4-5933e36769c5\") "
Nov 03 16:38:09.862935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.862902    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e1d654b-6a17-473c-bcb4-5933e36769c5-istiod-ca-cert" pod "4e1d654b-6a17-473c-bcb4-5933e36769c5" (UID: "4e1d654b-6a17-473c-bcb4-5933e36769c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:38:09.863180 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.863136    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-credential-socket" pod "4e1d654b-6a17-473c-bcb4-5933e36769c5" (UID: "4e1d654b-6a17-473c-bcb4-5933e36769c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:38:09.863687 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.863377    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.863829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.863696    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.863829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.863713    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e1d654b-6a17-473c-bcb4-5933e36769c5-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.863829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.863743    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.863829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.863757    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6d14b41-06d9-4fce-92fd-de9a147d3316-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.863829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.863772    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj8cp\" (UniqueName: \"kubernetes.io/projected/f6d14b41-06d9-4fce-92fd-de9a147d3316-kube-api-access-dj8cp\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.863829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.863786    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.863829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.863450    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-socket" pod "4e1d654b-6a17-473c-bcb4-5933e36769c5" (UID: "4e1d654b-6a17-473c-bcb4-5933e36769c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:38:09.863829 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.863633    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-data" pod "4e1d654b-6a17-473c-bcb4-5933e36769c5" (UID: "4e1d654b-6a17-473c-bcb4-5933e36769c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:38:09.871405 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.871377    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-kube-api-access-5nzxf" pod "4e1d654b-6a17-473c-bcb4-5933e36769c5" (UID: "4e1d654b-6a17-473c-bcb4-5933e36769c5"). InnerVolumeSpecName "kube-api-access-5nzxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:38:09.871508 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.871490    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-envoy" pod "4e1d654b-6a17-473c-bcb4-5933e36769c5" (UID: "4e1d654b-6a17-473c-bcb4-5933e36769c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:38:09.871802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.871776    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-podinfo" pod "4e1d654b-6a17-473c-bcb4-5933e36769c5" (UID: "4e1d654b-6a17-473c-bcb4-5933e36769c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:38:09.871968 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.871948    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-token" pod "4e1d654b-6a17-473c-bcb4-5933e36769c5" (UID: "4e1d654b-6a17-473c-bcb4-5933e36769c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:38:09.977688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.977283    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nzxf\" (UniqueName: \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-kube-api-access-5nzxf\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.977688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.977313    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.977688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.977329    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.977688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.977343    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.977688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.977354    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:09.977688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:09.977368    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e1d654b-6a17-473c-bcb4-5933e36769c5-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:38:10.151964 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:10.151931    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"]
Nov 03 16:38:10.157847 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:10.157819    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj"]
Nov 03 16:38:10.309671 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:10.309587    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" path="/var/lib/kubelet/pods/f6d14b41-06d9-4fce-92fd-de9a147d3316/volumes"
Nov 03 16:38:10.520577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:10.519196    5605 patch_prober.go:29] interesting pod/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:38:10.520577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:10.519283    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-59hnj" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:38:10.717691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:10.717655    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"
Nov 03 16:38:10.717691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:10.717668    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9" event={"ID":"4e1d654b-6a17-473c-bcb4-5933e36769c5","Type":"ContainerDied","Data":"f544a2f7404bb558b0d24b680042e223ba975549402e2019237f96b2fdd3fe81"}
Nov 03 16:38:10.718306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:10.717782    5605 scope.go:122] "RemoveContainer" containerID="aef0d03426c583f3bc4bac6ddf427a0b8e147891de38b3b8eec54f93e5b4561d"
Nov 03 16:38:10.974803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:10.974408    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"]
Nov 03 16:38:10.974803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:10.974486    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-jtrf9"]
Nov 03 16:38:12.309867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:12.309835    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" path="/var/lib/kubelet/pods/4e1d654b-6a17-473c-bcb4-5933e36769c5/volumes"
Nov 03 16:38:19.713886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:19.713853    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 16:38:19.714324 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:19.713939    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 16:38:25.844642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:25.844598    5605 generic.go:356] "Generic (PLEG): container finished" podID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerID="3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e" exitCode=0
Nov 03 16:38:25.845135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:25.844657    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" event={"ID":"a8c082b7-7db2-4f35-a62d-51750ca323ab","Type":"ContainerDied","Data":"3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e"}
Nov 03 16:38:25.845222 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:25.845202    5605 scope.go:122] "RemoveContainer" containerID="3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e"
Nov 03 16:38:25.847011 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:25.846980    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:38:25.847136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:25.847061    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:38:25.847136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:25.847107    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:38:26.853720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:26.853689    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" event={"ID":"a8c082b7-7db2-4f35-a62d-51750ca323ab","Type":"ContainerStarted","Data":"ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309"}
Nov 03 16:38:27.534100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:27.534065    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:38:27.534429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:27.534403    5605 patch_prober.go:29] interesting pod/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 03 16:38:27.534496 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:27.534467    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 03 16:38:28.534567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:28.534528    5605 patch_prober.go:29] interesting pod/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 03 16:38:28.535102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:28.534590    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 03 16:38:29.534104 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:29.534071    5605 patch_prober.go:29] interesting pod/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 03 16:38:29.534276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:29.534119    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 03 16:38:30.534562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:30.534528    5605 patch_prober.go:29] interesting pod/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 03 16:38:30.535014 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:30.534585    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 03 16:38:31.534054 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:31.534020    5605 patch_prober.go:29] interesting pod/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 03 16:38:31.534237 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:31.534090    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 03 16:38:32.235872 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.235833    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:38:32.301148 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.301114    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"]
Nov 03 16:38:32.301796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.301761    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy"
Nov 03 16:38:32.301796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.301777    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy"
Nov 03 16:38:32.301891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.301811    5605 cpu_manager.go:414] "containerMap: removing container" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerName="istio-proxy"
Nov 03 16:38:32.301891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.301817    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerName="istio-proxy"
Nov 03 16:38:32.301952 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.301914    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e1d654b-6a17-473c-bcb4-5933e36769c5" containerName="istio-proxy"
Nov 03 16:38:32.301952 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.301928    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6d14b41-06d9-4fce-92fd-de9a147d3316" containerName="istio-proxy"
Nov 03 16:38:32.304947 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.304928    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.418341 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.418305    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-d5937b-openshift-default-dockercfg-r2w8h\""
Nov 03 16:38:32.470252 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.441541    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"]
Nov 03 16:38:32.582802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.492835    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-envoy\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.582802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.579564    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-certs\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.582802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.579633    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-token\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.613754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.589971    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28dxn\" (UniqueName: \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-kube-api-access-28dxn\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.613754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.590026    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41871919-9219-49c2-abd8-6a38e7e32d2e-istiod-ca-cert\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.613754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.590058    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-podinfo\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.613754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.590120    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-credential-socket\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.613754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.590218    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-socket\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.613754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.590271    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-data\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.746831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.745784    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-certs\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.746831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.745860    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-token\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.746831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.746003    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28dxn\" (UniqueName: \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-kube-api-access-28dxn\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.746831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.746036    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41871919-9219-49c2-abd8-6a38e7e32d2e-istiod-ca-cert\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.746831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.746064    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-podinfo\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.746831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.746115    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-credential-socket\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.746831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.746198    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-socket\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.746831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.746246    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-data\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.746831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.746333    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-envoy\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.754784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.750942    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-certs\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.754784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.751692    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-socket\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.754784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.751949    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-data\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.754784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.752567    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41871919-9219-49c2-abd8-6a38e7e32d2e-istiod-ca-cert\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.906766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.903368    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-credential-socket\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.913457 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.912463    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-token\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.939817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.919891    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-podinfo\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.939817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.920554    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-envoy\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:32.939817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.921705    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:38:32.966762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:32.959343    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28dxn\" (UniqueName: \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-kube-api-access-28dxn\") pod \"e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") " pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:33.003263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:33.003042    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:38:33.107503 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:33.107467    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:33.400659 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:38:33.400621    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:38:33.546844 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:33.546813    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"]
Nov 03 16:38:33.547222 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:38:33.547198    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41871919_9219_49c2_abd8_6a38e7e32d2e.slice/crio-289e205c26cb6346f39319d3ee442961754a805bdf6a6dd0547f7ecd301ed107 WatchSource:0}: Error finding container 289e205c26cb6346f39319d3ee442961754a805bdf6a6dd0547f7ecd301ed107: Status 404 returned error can't find the container with id 289e205c26cb6346f39319d3ee442961754a805bdf6a6dd0547f7ecd301ed107
Nov 03 16:38:33.549778 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:33.549746    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:38:33.549862 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:33.549817    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:38:33.549862 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:33.549849    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:38:34.000880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:34.000841    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" event={"ID":"41871919-9219-49c2-abd8-6a38e7e32d2e","Type":"ContainerStarted","Data":"e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679"}
Nov 03 16:38:34.001118 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:34.000901    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" event={"ID":"41871919-9219-49c2-abd8-6a38e7e32d2e","Type":"ContainerStarted","Data":"289e205c26cb6346f39319d3ee442961754a805bdf6a6dd0547f7ecd301ed107"}
Nov 03 16:38:34.107801 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:34.107768    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:34.114164 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:34.114135    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:34.128583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:34.128535    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" podStartSLOduration=2.128519074 podStartE2EDuration="2.128519074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:38:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:38:34.125324138 +0000 UTC m=+1599.949775411" watchObservedRunningTime="2026-11-03 16:38:34.128519074 +0000 UTC m=+1599.952970346"
Nov 03 16:38:35.007463 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:35.007430    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:35.008453 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:35.008438    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:38:35.898914 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:35.898884    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"]
Nov 03 16:38:35.902373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:35.902358    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:35.974779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:35.968192    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-438248-openshift-default-dockercfg-n9fbj\""
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.060390    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-socket\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.060471    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-credential-socket\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.060540    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-data\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.060577    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84b165c2-3d89-452c-852b-ffa8ca079c72-istiod-ca-cert\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.060637    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpvft\" (UniqueName: \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-kube-api-access-jpvft\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.060757    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-token\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.060821    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-envoy\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.060887    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-podinfo\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.060934    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-certs\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.180008    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-token\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.180088    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-envoy\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.180147    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-podinfo\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.180191    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-certs\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.180279    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-socket\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.180336    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-credential-socket\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.180399    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-data\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.180433    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84b165c2-3d89-452c-852b-ffa8ca079c72-istiod-ca-cert\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.180492    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpvft\" (UniqueName: \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-kube-api-access-jpvft\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.210812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.191454    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-socket\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.248764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.224624    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-certs\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.248764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.224973    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-data\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.248764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.225261    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-credential-socket\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.248764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.225893    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84b165c2-3d89-452c-852b-ffa8ca079c72-istiod-ca-cert\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.411748 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.403537    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-token\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.494018 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.450530    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-podinfo\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.494018 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.486633    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpvft\" (UniqueName: \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-kube-api-access-jpvft\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.494018 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.487279    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-envoy\") pod \"e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") " pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.551758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.534867    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:36.610508 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:36.556670    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"]
Nov 03 16:38:37.238983 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:37.235564    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"]
Nov 03 16:38:37.273755 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:38:37.270306    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b165c2_3d89_452c_852b_ffa8ca079c72.slice/crio-b3ff156ac4bbbf969cb21db105fa0f1cfaa72c83e796b7a1e57551987f278405 WatchSource:0}: Error finding container b3ff156ac4bbbf969cb21db105fa0f1cfaa72c83e796b7a1e57551987f278405: Status 404 returned error can't find the container with id b3ff156ac4bbbf969cb21db105fa0f1cfaa72c83e796b7a1e57551987f278405
Nov 03 16:38:37.289902 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:37.289460    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:38:37.289902 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:37.289565    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:38:37.289902 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:37.289609    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:38:37.407563 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:37.407514    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" event={"ID":"84b165c2-3d89-452c-852b-ffa8ca079c72","Type":"ContainerStarted","Data":"1a56961bfa843ddf73915395d367b9cda57b21030184ddfc61d097f2a3fdc630"}
Nov 03 16:38:37.407720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:37.407574    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" event={"ID":"84b165c2-3d89-452c-852b-ffa8ca079c72","Type":"ContainerStarted","Data":"b3ff156ac4bbbf969cb21db105fa0f1cfaa72c83e796b7a1e57551987f278405"}
Nov 03 16:38:37.533080 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:37.532975    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" podStartSLOduration=2.5329611119999997 podStartE2EDuration="2.532961112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:38:37.531609113 +0000 UTC m=+1603.356060384" watchObservedRunningTime="2026-11-03 16:38:37.532961112 +0000 UTC m=+1603.357412382"
Nov 03 16:38:37.537892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:37.537869    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:38.544136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:38.544108    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:39.420062 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:39.420032    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:39.421105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:39.421082    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:38:44.080049 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.079950    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"]
Nov 03 16:38:44.089853 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.089826    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 16:38:44.131563 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.131532    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"]
Nov 03 16:38:44.136933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.136903    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0233caa0-0726-4bd8-a510-fbc1799b3eab-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw\" (UID: \"0233caa0-0726-4bd8-a510-fbc1799b3eab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 16:38:44.137182 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.137164    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxp22\" (UniqueName: \"kubernetes.io/projected/0233caa0-0726-4bd8-a510-fbc1799b3eab-kube-api-access-bxp22\") pod \"payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw\" (UID: \"0233caa0-0726-4bd8-a510-fbc1799b3eab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 16:38:44.178229 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.178197    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-d5937b\""
Nov 03 16:38:44.275762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.275701    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxp22\" (UniqueName: \"kubernetes.io/projected/0233caa0-0726-4bd8-a510-fbc1799b3eab-kube-api-access-bxp22\") pod \"payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw\" (UID: \"0233caa0-0726-4bd8-a510-fbc1799b3eab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 16:38:44.301757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.276178    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0233caa0-0726-4bd8-a510-fbc1799b3eab-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw\" (UID: \"0233caa0-0726-4bd8-a510-fbc1799b3eab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 16:38:44.305211 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.303517    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0233caa0-0726-4bd8-a510-fbc1799b3eab-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw\" (UID: \"0233caa0-0726-4bd8-a510-fbc1799b3eab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 16:38:44.551752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.549588    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxp22\" (UniqueName: \"kubernetes.io/projected/0233caa0-0726-4bd8-a510-fbc1799b3eab-kube-api-access-bxp22\") pod \"payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw\" (UID: \"0233caa0-0726-4bd8-a510-fbc1799b3eab\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 16:38:44.604657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.604623    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 16:38:44.663877 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.663386    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"]
Nov 03 16:38:44.670448 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.670424    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"]
Nov 03 16:38:44.670658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.670511    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 16:38:44.726362 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.726323    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-d5937b-dockercfg-8mc44\""
Nov 03 16:38:44.845897 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.845863    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9744816f-937a-43ab-b49d-8cfc4c4522dc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj\" (UID: \"9744816f-937a-43ab-b49d-8cfc4c4522dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 16:38:44.845897 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.845900    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9vd8\" (UniqueName: \"kubernetes.io/projected/9744816f-937a-43ab-b49d-8cfc4c4522dc-kube-api-access-c9vd8\") pod \"payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj\" (UID: \"9744816f-937a-43ab-b49d-8cfc4c4522dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 16:38:44.947904 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.947852    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9744816f-937a-43ab-b49d-8cfc4c4522dc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj\" (UID: \"9744816f-937a-43ab-b49d-8cfc4c4522dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 16:38:44.947904 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.947915    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9vd8\" (UniqueName: \"kubernetes.io/projected/9744816f-937a-43ab-b49d-8cfc4c4522dc-kube-api-access-c9vd8\") pod \"payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj\" (UID: \"9744816f-937a-43ab-b49d-8cfc4c4522dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 16:38:44.948637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:44.948617    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9744816f-937a-43ab-b49d-8cfc4c4522dc-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj\" (UID: \"9744816f-937a-43ab-b49d-8cfc4c4522dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 16:38:45.008608 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:45.008579    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9vd8\" (UniqueName: \"kubernetes.io/projected/9744816f-937a-43ab-b49d-8cfc4c4522dc-kube-api-access-c9vd8\") pod \"payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj\" (UID: \"9744816f-937a-43ab-b49d-8cfc4c4522dc\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 16:38:45.040124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:45.040096    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"]
Nov 03 16:38:45.042236 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:38:45.042203    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0233caa0_0726_4bd8_a510_fbc1799b3eab.slice/crio-5bc7d4f2836d983a62dd76d98e4bbb8cc6d9fae83c4292f7084cb25e8ad91f97 WatchSource:0}: Error finding container 5bc7d4f2836d983a62dd76d98e4bbb8cc6d9fae83c4292f7084cb25e8ad91f97: Status 404 returned error can't find the container with id 5bc7d4f2836d983a62dd76d98e4bbb8cc6d9fae83c4292f7084cb25e8ad91f97
Nov 03 16:38:45.285996 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:45.285967    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 16:38:45.674529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:45.674499    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"]
Nov 03 16:38:45.676355 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:38:45.676326    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9744816f_937a_43ab_b49d_8cfc4c4522dc.slice/crio-1369194df577cb48c9377bdca87097233f36e30a5aeedb1e8aaec9992e083d05 WatchSource:0}: Error finding container 1369194df577cb48c9377bdca87097233f36e30a5aeedb1e8aaec9992e083d05: Status 404 returned error can't find the container with id 1369194df577cb48c9377bdca87097233f36e30a5aeedb1e8aaec9992e083d05
Nov 03 16:38:45.678559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:45.678532    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw" event={"ID":"0233caa0-0726-4bd8-a510-fbc1799b3eab","Type":"ContainerStarted","Data":"5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688"}
Nov 03 16:38:45.678669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:45.678586    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw" event={"ID":"0233caa0-0726-4bd8-a510-fbc1799b3eab","Type":"ContainerStarted","Data":"5bc7d4f2836d983a62dd76d98e4bbb8cc6d9fae83c4292f7084cb25e8ad91f97"}
Nov 03 16:38:45.678669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:45.678650    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 16:38:45.802031 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:45.801977    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw" podStartSLOduration=1.801963453 podStartE2EDuration="1.801963453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:38:45.800896336 +0000 UTC m=+1611.625347618" watchObservedRunningTime="2026-11-03 16:38:45.801963453 +0000 UTC m=+1611.626414724"
Nov 03 16:38:46.687761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:46.687704    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj" event={"ID":"9744816f-937a-43ab-b49d-8cfc4c4522dc","Type":"ContainerStarted","Data":"6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834"}
Nov 03 16:38:46.688173 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:46.687788    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 16:38:46.688173 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:46.687813    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj" event={"ID":"9744816f-937a-43ab-b49d-8cfc4c4522dc","Type":"ContainerStarted","Data":"1369194df577cb48c9377bdca87097233f36e30a5aeedb1e8aaec9992e083d05"}
Nov 03 16:38:46.819745 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:46.819670    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj" podStartSLOduration=2.81964991 podStartE2EDuration="2.81964991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:38:46.816102372 +0000 UTC m=+1612.640553646" watchObservedRunningTime="2026-11-03 16:38:46.81964991 +0000 UTC m=+1612.644101181"
Nov 03 16:38:52.961697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:52.961657    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"]
Nov 03 16:38:52.980772 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:52.980743    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:38:53.057755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.055176    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"]
Nov 03 16:38:53.117749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.117566    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-438248-dockercfg-qn8fn\""
Nov 03 16:38:53.117749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.117668    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-438248\""
Nov 03 16:38:53.143806 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.119658    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm\" (UID: \"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:38:53.143806 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.119742    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx469\" (UniqueName: \"kubernetes.io/projected/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-kube-api-access-qx469\") pod \"payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm\" (UID: \"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:38:53.148748 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.147429    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"]
Nov 03 16:38:53.188203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.187263    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:38:53.222296 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.221829    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm\" (UID: \"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:38:53.222296 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.221893    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx469\" (UniqueName: \"kubernetes.io/projected/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-kube-api-access-qx469\") pod \"payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm\" (UID: \"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:38:53.237418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.237385    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm\" (UID: \"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:38:53.435049 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.426344    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205bb884-1493-4036-9e7d-5a995c5826cb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b\" (UID: \"205bb884-1493-4036-9e7d-5a995c5826cb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:38:53.435049 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.426503    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj4lf\" (UniqueName: \"kubernetes.io/projected/205bb884-1493-4036-9e7d-5a995c5826cb-kube-api-access-tj4lf\") pod \"payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b\" (UID: \"205bb884-1493-4036-9e7d-5a995c5826cb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:38:53.566013 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.538140    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205bb884-1493-4036-9e7d-5a995c5826cb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b\" (UID: \"205bb884-1493-4036-9e7d-5a995c5826cb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:38:53.566013 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.538291    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj4lf\" (UniqueName: \"kubernetes.io/projected/205bb884-1493-4036-9e7d-5a995c5826cb-kube-api-access-tj4lf\") pod \"payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b\" (UID: \"205bb884-1493-4036-9e7d-5a995c5826cb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:38:53.566013 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.539258    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205bb884-1493-4036-9e7d-5a995c5826cb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b\" (UID: \"205bb884-1493-4036-9e7d-5a995c5826cb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:38:53.608758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.608351    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"]
Nov 03 16:38:53.859245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.859178    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj4lf\" (UniqueName: \"kubernetes.io/projected/205bb884-1493-4036-9e7d-5a995c5826cb-kube-api-access-tj4lf\") pod \"payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b\" (UID: \"205bb884-1493-4036-9e7d-5a995c5826cb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:38:53.905749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.900568    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx469\" (UniqueName: \"kubernetes.io/projected/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-kube-api-access-qx469\") pod \"payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm\" (UID: \"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:38:53.936757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:53.930422    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:38:54.150814 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:54.149914    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:38:54.472106 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:54.471784    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"]
Nov 03 16:38:54.504768 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:38:54.501407    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fcf4a33_5dcb_4573_9581_ad772f3d2ebc.slice/crio-5a81f43eb4abe48039a6008774bfd7962ecf051c03315c566aec90a9a7f98917 WatchSource:0}: Error finding container 5a81f43eb4abe48039a6008774bfd7962ecf051c03315c566aec90a9a7f98917: Status 404 returned error can't find the container with id 5a81f43eb4abe48039a6008774bfd7962ecf051c03315c566aec90a9a7f98917
Nov 03 16:38:55.203892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.203834    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 16:38:55.456988 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.454124    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:38:55.487754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.486822    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:38:55.487754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.487316    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": context deadline exceeded"
Nov 03 16:38:55.508755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.498836    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:38:55.508755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.499535    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Nov 03 16:38:55.596919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.593270    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:38:55.596919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.595302    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:38:55.668770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.619347    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:38:55.699503 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.621105    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld" podUID="f237b2f1-9a28-419e-805a-0d77cbda7230" containerName="main" probeResult="failure" output="Get \"https://10.133.0.27:8000/ready\": context deadline exceeded"
Nov 03 16:38:55.704791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.621152    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/ready\": context deadline exceeded"
Nov 03 16:38:55.774841 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:55.773320    5605 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:38:56.123768 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:56.088462    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"]
Nov 03 16:38:56.210279 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:56.210237    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm" event={"ID":"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc","Type":"ContainerStarted","Data":"5a81f43eb4abe48039a6008774bfd7962ecf051c03315c566aec90a9a7f98917"}
Nov 03 16:38:56.212326 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:56.210835    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:38:56.212326 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:56.211249    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:38:56.247635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:56.247593    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rn227 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:38:56.247832 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:56.247679    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" podUID="b0246a86-f897-42f0-88eb-38c00c8b1bc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:38:56.299370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:56.299344    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 16:38:56.690543 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:56.690469    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 16:38:57.236646 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:57.236605    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm" event={"ID":"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc","Type":"ContainerStarted","Data":"4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900"}
Nov 03 16:38:57.237163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:57.236710    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:38:57.238471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:57.238443    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b" event={"ID":"205bb884-1493-4036-9e7d-5a995c5826cb","Type":"ContainerStarted","Data":"4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8"}
Nov 03 16:38:57.238607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:57.238493    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b" event={"ID":"205bb884-1493-4036-9e7d-5a995c5826cb","Type":"ContainerStarted","Data":"d16d40fe0bf5c84fb6afbcceefa4b15f9795a09066b8fcae78a77edbef3a0e68"}
Nov 03 16:38:57.238670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:57.238632    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:38:57.358839 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:57.358790    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm" podStartSLOduration=5.35877605 podStartE2EDuration="5.35877605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:38:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:38:57.357659926 +0000 UTC m=+1623.182111196" watchObservedRunningTime="2026-11-03 16:38:57.35877605 +0000 UTC m=+1623.183227321"
Nov 03 16:38:57.535963 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:57.531260    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b" podStartSLOduration=4.531235223 podStartE2EDuration="4.531235223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:38:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:38:57.519815172 +0000 UTC m=+1623.344266444" watchObservedRunningTime="2026-11-03 16:38:57.531235223 +0000 UTC m=+1623.355686496"
Nov 03 16:38:57.714254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:57.712247    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 16:38:58.944798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:58.944762    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"]
Nov 03 16:38:59.071489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:38:59.071364    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"]
Nov 03 16:39:03.393888 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:03.393853    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:39:03.945519 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:03.945467    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745" podUID="896b4895-c8a7-4fb8-960d-22a0603a8a34" containerName="payload-processing" containerID="cri-o://8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2" gracePeriod=25
Nov 03 16:39:04.072181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.072123    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4" podUID="11f0533a-48e2-49d7-8106-6d1c40baa827" containerName="payload-pre-processing" containerID="cri-o://c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1" gracePeriod=25
Nov 03 16:39:04.252217 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.252183    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:39:04.365608 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.365583    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:39:04.377913 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.377884    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/896b4895-c8a7-4fb8-960d-22a0603a8a34-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896b4895-c8a7-4fb8-960d-22a0603a8a34-plugins-config-volume\") pod \"896b4895-c8a7-4fb8-960d-22a0603a8a34\" (UID: \"896b4895-c8a7-4fb8-960d-22a0603a8a34\") "
Nov 03 16:39:04.378080 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.378063    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/896b4895-c8a7-4fb8-960d-22a0603a8a34-kube-api-access-mnpw9\" (UniqueName: \"kubernetes.io/projected/896b4895-c8a7-4fb8-960d-22a0603a8a34-kube-api-access-mnpw9\") pod \"896b4895-c8a7-4fb8-960d-22a0603a8a34\" (UID: \"896b4895-c8a7-4fb8-960d-22a0603a8a34\") "
Nov 03 16:39:04.378279 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.378256    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/896b4895-c8a7-4fb8-960d-22a0603a8a34-plugins-config-volume" pod "896b4895-c8a7-4fb8-960d-22a0603a8a34" (UID: "896b4895-c8a7-4fb8-960d-22a0603a8a34"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:04.378990 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.378971    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896b4895-c8a7-4fb8-960d-22a0603a8a34-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:04.380331 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.380310    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/896b4895-c8a7-4fb8-960d-22a0603a8a34-kube-api-access-mnpw9" pod "896b4895-c8a7-4fb8-960d-22a0603a8a34" (UID: "896b4895-c8a7-4fb8-960d-22a0603a8a34"). InnerVolumeSpecName "kube-api-access-mnpw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:04.479971 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.479880    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11f0533a-48e2-49d7-8106-6d1c40baa827-kube-api-access-5z7b4\" (UniqueName: \"kubernetes.io/projected/11f0533a-48e2-49d7-8106-6d1c40baa827-kube-api-access-5z7b4\") pod \"11f0533a-48e2-49d7-8106-6d1c40baa827\" (UID: \"11f0533a-48e2-49d7-8106-6d1c40baa827\") "
Nov 03 16:39:04.480449 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.480065    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11f0533a-48e2-49d7-8106-6d1c40baa827-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f0533a-48e2-49d7-8106-6d1c40baa827-plugins-config-volume\") pod \"11f0533a-48e2-49d7-8106-6d1c40baa827\" (UID: \"11f0533a-48e2-49d7-8106-6d1c40baa827\") "
Nov 03 16:39:04.480518 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.480493    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11f0533a-48e2-49d7-8106-6d1c40baa827-plugins-config-volume" pod "11f0533a-48e2-49d7-8106-6d1c40baa827" (UID: "11f0533a-48e2-49d7-8106-6d1c40baa827"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:04.480949 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.480921    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f0533a-48e2-49d7-8106-6d1c40baa827-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:04.480949 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.480943    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnpw9\" (UniqueName: \"kubernetes.io/projected/896b4895-c8a7-4fb8-960d-22a0603a8a34-kube-api-access-mnpw9\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:04.482346 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.482324    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f0533a-48e2-49d7-8106-6d1c40baa827-kube-api-access-5z7b4" pod "11f0533a-48e2-49d7-8106-6d1c40baa827" (UID: "11f0533a-48e2-49d7-8106-6d1c40baa827"). InnerVolumeSpecName "kube-api-access-5z7b4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:04.488478 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.488455    5605 generic.go:356] "Generic (PLEG): container finished" podID="11f0533a-48e2-49d7-8106-6d1c40baa827" containerID="c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1" exitCode=0
Nov 03 16:39:04.488586 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.488499    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"
Nov 03 16:39:04.488586 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.488535    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4" event={"ID":"11f0533a-48e2-49d7-8106-6d1c40baa827","Type":"ContainerDied","Data":"c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1"}
Nov 03 16:39:04.488668 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.488589    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4" event={"ID":"11f0533a-48e2-49d7-8106-6d1c40baa827","Type":"ContainerDied","Data":"a7fa0bde15be8b62ba79b9c6d08a1a7de1c69f4428e0693e0e96419bcdb7e4cf"}
Nov 03 16:39:04.488668 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.488612    5605 scope.go:122] "RemoveContainer" containerID="c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1"
Nov 03 16:39:04.490177 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.490152    5605 generic.go:356] "Generic (PLEG): container finished" podID="896b4895-c8a7-4fb8-960d-22a0603a8a34" containerID="8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2" exitCode=0
Nov 03 16:39:04.490293 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.490182    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745" event={"ID":"896b4895-c8a7-4fb8-960d-22a0603a8a34","Type":"ContainerDied","Data":"8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2"}
Nov 03 16:39:04.490293 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.490209    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"
Nov 03 16:39:04.490293 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.490227    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745" event={"ID":"896b4895-c8a7-4fb8-960d-22a0603a8a34","Type":"ContainerDied","Data":"7e0e00f2f86780acc4c50f079e38e2e85f33dd23837709723ca275dc4ab40a91"}
Nov 03 16:39:04.500051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.499960    5605 scope.go:122] "RemoveContainer" containerID="c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1"
Nov 03 16:39:04.500359 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:04.500340    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1\": container with ID starting with c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1 not found: ID does not exist" containerID="c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1"
Nov 03 16:39:04.500432 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.500368    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1"} err="failed to get container status \"c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1\": rpc error: code = NotFound desc = could not find container \"c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1\": container with ID starting with c6eaf82b2d89a20b1fbd8de18666bd793f1fcd43fe6b6c370cea1f5550007ac1 not found: ID does not exist"
Nov 03 16:39:04.500432 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.500386    5605 scope.go:122] "RemoveContainer" containerID="8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2"
Nov 03 16:39:04.510297 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.510272    5605 scope.go:122] "RemoveContainer" containerID="8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2"
Nov 03 16:39:04.510591 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:04.510573    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2\": container with ID starting with 8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2 not found: ID does not exist" containerID="8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2"
Nov 03 16:39:04.510641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.510600    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2"} err="failed to get container status \"8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2\": rpc error: code = NotFound desc = could not find container \"8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2\": container with ID starting with 8bed94d7c7d063d79327d51096e9260efeffbc5d2ec2a22054cfa787e97a1df2 not found: ID does not exist"
Nov 03 16:39:04.582990 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.582952    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z7b4\" (UniqueName: \"kubernetes.io/projected/11f0533a-48e2-49d7-8106-6d1c40baa827-kube-api-access-5z7b4\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:04.699819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.699787    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"]
Nov 03 16:39:04.706810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.706784    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b70a8a53-5bbbc58c57-z7745"]
Nov 03 16:39:04.821115 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.821023    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"]
Nov 03 16:39:04.828720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:04.828691    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4"]
Nov 03 16:39:05.341359 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:05.341319    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-b70a8a53-678c4c9ddc-sxxz4" podUID="11f0533a-48e2-49d7-8106-6d1c40baa827" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.84:50052\" within 1s: context deadline exceeded"
Nov 03 16:39:05.341545 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:39:05.341390    5605 logging.go:55] [core] [Channel #224 SubChannel #225]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.84:50052", ServerName: "10.133.0.84:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.84:50052: operation was canceled"
Nov 03 16:39:06.312618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:06.312589    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11f0533a-48e2-49d7-8106-6d1c40baa827" path="/var/lib/kubelet/pods/11f0533a-48e2-49d7-8106-6d1c40baa827/volumes"
Nov 03 16:39:06.313056 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:06.313041    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="896b4895-c8a7-4fb8-960d-22a0603a8a34" path="/var/lib/kubelet/pods/896b4895-c8a7-4fb8-960d-22a0603a8a34/volumes"
Nov 03 16:39:08.248186 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:08.248153    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:39:08.248605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:08.248351    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:39:18.511006 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:18.510970    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"]
Nov 03 16:39:18.511501 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:18.511192    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy" containerID="cri-o://904878db9d16bd72b76fdcab1ff80f8ed54d7475c9e8897afadbf144cf4e6ad4" gracePeriod=30
Nov 03 16:39:18.564368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:18.564331    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"]
Nov 03 16:39:18.564616 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:18.564576    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy" containerID="cri-o://7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef" gracePeriod=30
Nov 03 16:39:18.568092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:18.568065    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"]
Nov 03 16:39:18.568357 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:18.568292    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy" containerID="cri-o://87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c" gracePeriod=30
Nov 03 16:39:18.613719 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:18.613685    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"]
Nov 03 16:39:18.613920 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:18.613880    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy" containerID="cri-o://009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377" gracePeriod=30
Nov 03 16:39:19.864065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:19.864021    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"]
Nov 03 16:39:19.864561 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:19.864269    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" containerID="cri-o://a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966" gracePeriod=30
Nov 03 16:39:19.915029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:19.914992    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"]
Nov 03 16:39:19.915303 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:19.915273    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy" containerID="cri-o://f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b" gracePeriod=30
Nov 03 16:39:19.919421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:19.919392    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"]
Nov 03 16:39:19.919924 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:19.919888    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" containerID="cri-o://b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80" gracePeriod=30
Nov 03 16:39:19.966209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:19.966175    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"]
Nov 03 16:39:19.966443 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:19.966419    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy" containerID="cri-o://223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4" gracePeriod=30
Nov 03 16:39:20.216017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:20.215940    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:39:20.216017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:20.215995    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:39:21.303902 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:21.303866    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:39:21.304296 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:21.303924    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:39:21.837382 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:21.837342    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:39:21.837551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:21.837416    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:39:22.024997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:22.024964    5605 patch_prober.go:29] interesting pod/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:39:22.025155 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:22.025026    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:39:23.516547 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.516398    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:39:23.516977 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.516602    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:39:23.647292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.647260    5605 generic.go:356] "Generic (PLEG): container finished" podID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerID="904878db9d16bd72b76fdcab1ff80f8ed54d7475c9e8897afadbf144cf4e6ad4" exitCode=0
Nov 03 16:39:23.647475 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.647340    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" event={"ID":"8584968f-9bed-42ab-bf5c-9d238e2ca0cd","Type":"ContainerDied","Data":"904878db9d16bd72b76fdcab1ff80f8ed54d7475c9e8897afadbf144cf4e6ad4"}
Nov 03 16:39:23.873914 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.873887    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:39:23.936122 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.936092    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:39:23.989892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.989712    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istiod-ca-cert\") pod \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") "
Nov 03 16:39:23.990016 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.989976    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-socket\") pod \"231bee41-ee7a-4b55-ad4c-4b39a0104869\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") "
Nov 03 16:39:23.990096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990081    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-data\") pod \"231bee41-ee7a-4b55-ad4c-4b39a0104869\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") "
Nov 03 16:39:23.990140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990125    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-data\") pod \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") "
Nov 03 16:39:23.990174 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990163    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-podinfo\") pod \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") "
Nov 03 16:39:23.990248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990230    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-certs\") pod \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") "
Nov 03 16:39:23.990384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990357    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-kube-api-access-lvw9j\" (UniqueName: \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-kube-api-access-lvw9j\") pod \"231bee41-ee7a-4b55-ad4c-4b39a0104869\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") "
Nov 03 16:39:23.990462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990403    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-token\" (UniqueName: \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-token\") pod \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") "
Nov 03 16:39:23.990462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990433    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-credential-socket\") pod \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") "
Nov 03 16:39:23.990538 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990470    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/231bee41-ee7a-4b55-ad4c-4b39a0104869-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231bee41-ee7a-4b55-ad4c-4b39a0104869-istiod-ca-cert\") pod \"231bee41-ee7a-4b55-ad4c-4b39a0104869\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") "
Nov 03 16:39:23.990538 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990504    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-kube-api-access-fbf5v\" (UniqueName: \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-kube-api-access-fbf5v\") pod \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") "
Nov 03 16:39:23.990538 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990513    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-data" pod "231bee41-ee7a-4b55-ad4c-4b39a0104869" (UID: "231bee41-ee7a-4b55-ad4c-4b39a0104869"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:23.990638 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990620    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-credential-socket\") pod \"231bee41-ee7a-4b55-ad4c-4b39a0104869\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") "
Nov 03 16:39:23.990676 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990661    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-certs\") pod \"231bee41-ee7a-4b55-ad4c-4b39a0104869\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") "
Nov 03 16:39:23.990811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990791    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-envoy\") pod \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") "
Nov 03 16:39:23.990889 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990835    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-podinfo\") pod \"231bee41-ee7a-4b55-ad4c-4b39a0104869\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") "
Nov 03 16:39:23.990889 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990837    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istiod-ca-cert" pod "8584968f-9bed-42ab-bf5c-9d238e2ca0cd" (UID: "8584968f-9bed-42ab-bf5c-9d238e2ca0cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:23.990999 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990890    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-envoy\") pod \"231bee41-ee7a-4b55-ad4c-4b39a0104869\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") "
Nov 03 16:39:23.990999 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990960    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-socket\") pod \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\" (UID: \"8584968f-9bed-42ab-bf5c-9d238e2ca0cd\") "
Nov 03 16:39:23.991075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.990998    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-token\" (UniqueName: \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-token\") pod \"231bee41-ee7a-4b55-ad4c-4b39a0104869\" (UID: \"231bee41-ee7a-4b55-ad4c-4b39a0104869\") "
Nov 03 16:39:23.991075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.991026    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-socket" pod "231bee41-ee7a-4b55-ad4c-4b39a0104869" (UID: "231bee41-ee7a-4b55-ad4c-4b39a0104869"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:23.991441 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.991383    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-credential-socket" pod "231bee41-ee7a-4b55-ad4c-4b39a0104869" (UID: "231bee41-ee7a-4b55-ad4c-4b39a0104869"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:23.991548 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.991446    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-credential-socket" pod "8584968f-9bed-42ab-bf5c-9d238e2ca0cd" (UID: "8584968f-9bed-42ab-bf5c-9d238e2ca0cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:23.991548 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.991460    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-data" pod "8584968f-9bed-42ab-bf5c-9d238e2ca0cd" (UID: "8584968f-9bed-42ab-bf5c-9d238e2ca0cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:23.992546 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.991804    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231bee41-ee7a-4b55-ad4c-4b39a0104869-istiod-ca-cert" pod "231bee41-ee7a-4b55-ad4c-4b39a0104869" (UID: "231bee41-ee7a-4b55-ad4c-4b39a0104869"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:23.992546 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.991861    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-certs" pod "8584968f-9bed-42ab-bf5c-9d238e2ca0cd" (UID: "8584968f-9bed-42ab-bf5c-9d238e2ca0cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:23.992546 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.992374    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-certs" pod "231bee41-ee7a-4b55-ad4c-4b39a0104869" (UID: "231bee41-ee7a-4b55-ad4c-4b39a0104869"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:23.993103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.992811    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-socket" pod "8584968f-9bed-42ab-bf5c-9d238e2ca0cd" (UID: "8584968f-9bed-42ab-bf5c-9d238e2ca0cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:23.993103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.992952    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:23.993103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.992978    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:23.993103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.992992    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:23.993103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.993006    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:23.993103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.993019    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:23.993103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.993034    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:23.993103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.993048    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/231bee41-ee7a-4b55-ad4c-4b39a0104869-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:23.993103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.993062    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:23.993103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.993075    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:23.995520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.995494    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-podinfo" pod "8584968f-9bed-42ab-bf5c-9d238e2ca0cd" (UID: "8584968f-9bed-42ab-bf5c-9d238e2ca0cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:23.995635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.995550    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-podinfo" pod "231bee41-ee7a-4b55-ad4c-4b39a0104869" (UID: "231bee41-ee7a-4b55-ad4c-4b39a0104869"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:23.995952 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.995929    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-kube-api-access-fbf5v" pod "8584968f-9bed-42ab-bf5c-9d238e2ca0cd" (UID: "8584968f-9bed-42ab-bf5c-9d238e2ca0cd"). InnerVolumeSpecName "kube-api-access-fbf5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:23.996097 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.996075    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-envoy" pod "8584968f-9bed-42ab-bf5c-9d238e2ca0cd" (UID: "8584968f-9bed-42ab-bf5c-9d238e2ca0cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:23.996413 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.996388    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-token" pod "231bee41-ee7a-4b55-ad4c-4b39a0104869" (UID: "231bee41-ee7a-4b55-ad4c-4b39a0104869"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:23.996625 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.996606    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-token" pod "8584968f-9bed-42ab-bf5c-9d238e2ca0cd" (UID: "8584968f-9bed-42ab-bf5c-9d238e2ca0cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:23.996690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.996637    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-kube-api-access-lvw9j" pod "231bee41-ee7a-4b55-ad4c-4b39a0104869" (UID: "231bee41-ee7a-4b55-ad4c-4b39a0104869"). InnerVolumeSpecName "kube-api-access-lvw9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:23.997032 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:23.997010    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-envoy" pod "231bee41-ee7a-4b55-ad4c-4b39a0104869" (UID: "231bee41-ee7a-4b55-ad4c-4b39a0104869"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.001520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.001499    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:39:24.004868 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.004849    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:39:24.093873 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.093835    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-credential-socket\") pod \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") "
Nov 03 16:39:24.094058 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.093895    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-certs\") pod \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") "
Nov 03 16:39:24.094058 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094001    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-data\") pod \"a69b3be4-c69f-48e6-881a-436892b08ec3\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094077    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-envoy\") pod \"a69b3be4-c69f-48e6-881a-436892b08ec3\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094117    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-envoy\") pod \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094124    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-credential-socket" pod "806a66bb-48eb-41ae-8fbe-c7869b6040f5" (UID: "806a66bb-48eb-41ae-8fbe-c7869b6040f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094166    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-token\" (UniqueName: \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-token\") pod \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094249    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istiod-ca-cert\") pod \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094279    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-certs\") pod \"a69b3be4-c69f-48e6-881a-436892b08ec3\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094282    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-certs" pod "806a66bb-48eb-41ae-8fbe-c7869b6040f5" (UID: "806a66bb-48eb-41ae-8fbe-c7869b6040f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094311    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-data" pod "a69b3be4-c69f-48e6-881a-436892b08ec3" (UID: "a69b3be4-c69f-48e6-881a-436892b08ec3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094379    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-kube-api-access-c6td6\" (UniqueName: \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-kube-api-access-c6td6\") pod \"a69b3be4-c69f-48e6-881a-436892b08ec3\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094418    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-socket\") pod \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094454    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-podinfo\") pod \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094482    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-token\" (UniqueName: \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-token\") pod \"a69b3be4-c69f-48e6-881a-436892b08ec3\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094572    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-kube-api-access-sjcl8\" (UniqueName: \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-kube-api-access-sjcl8\") pod \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094611    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-podinfo\") pod \"a69b3be4-c69f-48e6-881a-436892b08ec3\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094683    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-credential-socket\") pod \"a69b3be4-c69f-48e6-881a-436892b08ec3\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094739    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istiod-ca-cert" pod "806a66bb-48eb-41ae-8fbe-c7869b6040f5" (UID: "806a66bb-48eb-41ae-8fbe-c7869b6040f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094759    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-data\") pod \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\" (UID: \"806a66bb-48eb-41ae-8fbe-c7869b6040f5\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094807    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-socket\") pod \"a69b3be4-c69f-48e6-881a-436892b08ec3\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") "
Nov 03 16:39:24.094861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.094836    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a69b3be4-c69f-48e6-881a-436892b08ec3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a69b3be4-c69f-48e6-881a-436892b08ec3-istiod-ca-cert\") pod \"a69b3be4-c69f-48e6-881a-436892b08ec3\" (UID: \"a69b3be4-c69f-48e6-881a-436892b08ec3\") "
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095647    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095670    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095686    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095692    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-credential-socket" pod "a69b3be4-c69f-48e6-881a-436892b08ec3" (UID: "a69b3be4-c69f-48e6-881a-436892b08ec3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095699    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095756    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095772    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095786    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095801    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvw9j\" (UniqueName: \"kubernetes.io/projected/231bee41-ee7a-4b55-ad4c-4b39a0104869-kube-api-access-lvw9j\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095816    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095829    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbf5v\" (UniqueName: \"kubernetes.io/projected/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-kube-api-access-fbf5v\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095843    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095856    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8584968f-9bed-42ab-bf5c-9d238e2ca0cd-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.095953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.095869    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/231bee41-ee7a-4b55-ad4c-4b39a0104869-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.096643 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.096062    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a69b3be4-c69f-48e6-881a-436892b08ec3-istiod-ca-cert" pod "a69b3be4-c69f-48e6-881a-436892b08ec3" (UID: "a69b3be4-c69f-48e6-881a-436892b08ec3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:24.096643 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.096304    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-data" pod "806a66bb-48eb-41ae-8fbe-c7869b6040f5" (UID: "806a66bb-48eb-41ae-8fbe-c7869b6040f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.096643 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.096481    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-socket" pod "a69b3be4-c69f-48e6-881a-436892b08ec3" (UID: "a69b3be4-c69f-48e6-881a-436892b08ec3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.096842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.096702    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-certs" pod "a69b3be4-c69f-48e6-881a-436892b08ec3" (UID: "a69b3be4-c69f-48e6-881a-436892b08ec3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.097476 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.097442    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-socket" pod "806a66bb-48eb-41ae-8fbe-c7869b6040f5" (UID: "806a66bb-48eb-41ae-8fbe-c7869b6040f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.100247 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.100173    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-envoy" pod "a69b3be4-c69f-48e6-881a-436892b08ec3" (UID: "a69b3be4-c69f-48e6-881a-436892b08ec3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.100337 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.100264    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-podinfo" pod "a69b3be4-c69f-48e6-881a-436892b08ec3" (UID: "a69b3be4-c69f-48e6-881a-436892b08ec3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:24.100337 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.100288    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-token" pod "806a66bb-48eb-41ae-8fbe-c7869b6040f5" (UID: "806a66bb-48eb-41ae-8fbe-c7869b6040f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:24.100417 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.100396    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-envoy" pod "806a66bb-48eb-41ae-8fbe-c7869b6040f5" (UID: "806a66bb-48eb-41ae-8fbe-c7869b6040f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:24.100649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.100625    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-token" pod "a69b3be4-c69f-48e6-881a-436892b08ec3" (UID: "a69b3be4-c69f-48e6-881a-436892b08ec3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:24.100783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.100757    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-podinfo" pod "806a66bb-48eb-41ae-8fbe-c7869b6040f5" (UID: "806a66bb-48eb-41ae-8fbe-c7869b6040f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:24.100952 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.100925    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-kube-api-access-c6td6" pod "a69b3be4-c69f-48e6-881a-436892b08ec3" (UID: "a69b3be4-c69f-48e6-881a-436892b08ec3"). InnerVolumeSpecName "kube-api-access-c6td6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:24.101170 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.101151    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-kube-api-access-sjcl8" pod "806a66bb-48eb-41ae-8fbe-c7869b6040f5" (UID: "806a66bb-48eb-41ae-8fbe-c7869b6040f5"). InnerVolumeSpecName "kube-api-access-sjcl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:24.216048 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216010    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216048 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216047    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216068    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216082    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216097    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a69b3be4-c69f-48e6-881a-436892b08ec3-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216111    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216128    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216141    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216154    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a69b3be4-c69f-48e6-881a-436892b08ec3-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216167    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6td6\" (UniqueName: \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-kube-api-access-c6td6\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216187    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/806a66bb-48eb-41ae-8fbe-c7869b6040f5-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216200    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/806a66bb-48eb-41ae-8fbe-c7869b6040f5-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216213    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a69b3be4-c69f-48e6-881a-436892b08ec3-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.216283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.216226    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjcl8\" (UniqueName: \"kubernetes.io/projected/806a66bb-48eb-41ae-8fbe-c7869b6040f5-kube-api-access-sjcl8\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:24.716752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.715907    5605 generic.go:356] "Generic (PLEG): container finished" podID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerID="009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377" exitCode=0
Nov 03 16:39:24.716752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.716076    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" event={"ID":"806a66bb-48eb-41ae-8fbe-c7869b6040f5","Type":"ContainerDied","Data":"009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377"}
Nov 03 16:39:24.716752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.716124    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf" event={"ID":"806a66bb-48eb-41ae-8fbe-c7869b6040f5","Type":"ContainerDied","Data":"65b79546d2534b60c153954d105b1b4f8f5ce567f6c6b80a5eadc3bd9189362b"}
Nov 03 16:39:24.716752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.716149    5605 scope.go:122] "RemoveContainer" containerID="009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377"
Nov 03 16:39:24.716752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.716318    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"
Nov 03 16:39:24.729719 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.726072    5605 generic.go:356] "Generic (PLEG): container finished" podID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerID="87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c" exitCode=0
Nov 03 16:39:24.729719 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.726350    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" event={"ID":"a69b3be4-c69f-48e6-881a-436892b08ec3","Type":"ContainerDied","Data":"87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c"}
Nov 03 16:39:24.729719 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.726419    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w" event={"ID":"a69b3be4-c69f-48e6-881a-436892b08ec3","Type":"ContainerDied","Data":"cfb8a1d9cbb32f52d8bc3b187cd94eea6098ef46ff6c7d06e71032ab8f687e56"}
Nov 03 16:39:24.729719 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.726547    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"
Nov 03 16:39:24.743895 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.743860    5605 generic.go:356] "Generic (PLEG): container finished" podID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerID="7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef" exitCode=0
Nov 03 16:39:24.745194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.744163    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" event={"ID":"231bee41-ee7a-4b55-ad4c-4b39a0104869","Type":"ContainerDied","Data":"7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef"}
Nov 03 16:39:24.745194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.744361    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg" event={"ID":"231bee41-ee7a-4b55-ad4c-4b39a0104869","Type":"ContainerDied","Data":"368aa2e9428aca10cd686d50d041b90afa581d1db23a45e82f1afab14f27ebf1"}
Nov 03 16:39:24.745194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.744504    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"
Nov 03 16:39:24.758556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.758522    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm" event={"ID":"8584968f-9bed-42ab-bf5c-9d238e2ca0cd","Type":"ContainerDied","Data":"013559f844766ecca8a13475c8d82cdead7bd718062da906bb0e4099ae3bf91c"}
Nov 03 16:39:24.758919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.758898    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"
Nov 03 16:39:24.811687 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.811662    5605 scope.go:122] "RemoveContainer" containerID="5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d"
Nov 03 16:39:24.890129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.890100    5605 scope.go:122] "RemoveContainer" containerID="009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377"
Nov 03 16:39:24.890824 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:24.890794    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377\": container with ID starting with 009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377 not found: ID does not exist" containerID="009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377"
Nov 03 16:39:24.890985 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.890960    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377"} err="failed to get container status \"009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377\": rpc error: code = NotFound desc = could not find container \"009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377\": container with ID starting with 009d5546b641b05b00d94fe20398200a0641157fd4a00718e54485b40a8f5377 not found: ID does not exist"
Nov 03 16:39:24.891111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.891099    5605 scope.go:122] "RemoveContainer" containerID="5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d"
Nov 03 16:39:24.894751 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:24.893892    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d\": container with ID starting with 5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d not found: ID does not exist" containerID="5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d"
Nov 03 16:39:24.894751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.893958    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d"} err="failed to get container status \"5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d\": rpc error: code = NotFound desc = could not find container \"5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d\": container with ID starting with 5c9d99178264ce2f494257d7ac86dd484a88ff240885c26da448ef5708e33c5d not found: ID does not exist"
Nov 03 16:39:24.894751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.894002    5605 scope.go:122] "RemoveContainer" containerID="87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c"
Nov 03 16:39:24.935552 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.933819    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"]
Nov 03 16:39:24.938841 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:24.938809    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-jnwbf"]
Nov 03 16:39:25.144436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.140220    5605 scope.go:122] "RemoveContainer" containerID="87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c"
Nov 03 16:39:25.144436 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:25.142418    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c\": container with ID starting with 87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c not found: ID does not exist" containerID="87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c"
Nov 03 16:39:25.144436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.142461    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c"} err="failed to get container status \"87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c\": rpc error: code = NotFound desc = could not find container \"87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c\": container with ID starting with 87419aa287aa264b37ddfe2326bfc9631499141cd0a69f7310ecfb661bd2905c not found: ID does not exist"
Nov 03 16:39:25.144436 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.142489    5605 scope.go:122] "RemoveContainer" containerID="7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef"
Nov 03 16:39:25.173623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.173600    5605 scope.go:122] "RemoveContainer" containerID="7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef"
Nov 03 16:39:25.174050 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:25.174027    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef\": container with ID starting with 7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef not found: ID does not exist" containerID="7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef"
Nov 03 16:39:25.174185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.174061    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef"} err="failed to get container status \"7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef\": rpc error: code = NotFound desc = could not find container \"7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef\": container with ID starting with 7dfccb9c01d2ab169660baa571ca18fe2344cc44f65e6fa9930e8d705bb4a8ef not found: ID does not exist"
Nov 03 16:39:25.174185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.174090    5605 scope.go:122] "RemoveContainer" containerID="904878db9d16bd72b76fdcab1ff80f8ed54d7475c9e8897afadbf144cf4e6ad4"
Nov 03 16:39:25.175979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.175953    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:39:25.212774 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.212745    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"]
Nov 03 16:39:25.222093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.220498    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-bxd6w"]
Nov 03 16:39:25.242978 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.242948    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:39:25.331231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.330570    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:39:25.352835 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.352799    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-data\") pod \"e21a973c-98db-465f-b02e-f8e961c420ec\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") "
Nov 03 16:39:25.354913 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.354887    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-token\" (UniqueName: \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-token\") pod \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") "
Nov 03 16:39:25.355112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.354932    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istiod-ca-cert\") pod \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") "
Nov 03 16:39:25.355112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.354988    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-kube-api-access-z4mqd\" (UniqueName: \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-kube-api-access-z4mqd\") pod \"e21a973c-98db-465f-b02e-f8e961c420ec\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") "
Nov 03 16:39:25.355112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355048    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-credential-socket\") pod \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") "
Nov 03 16:39:25.355112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355073    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-data" pod "e21a973c-98db-465f-b02e-f8e961c420ec" (UID: "e21a973c-98db-465f-b02e-f8e961c420ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.355112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355108    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e21a973c-98db-465f-b02e-f8e961c420ec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e21a973c-98db-465f-b02e-f8e961c420ec-istio-podinfo\") pod \"e21a973c-98db-465f-b02e-f8e961c420ec\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") "
Nov 03 16:39:25.355422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355149    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-credential-socket\") pod \"e21a973c-98db-465f-b02e-f8e961c420ec\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") "
Nov 03 16:39:25.355422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355216    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-envoy\") pod \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") "
Nov 03 16:39:25.355422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355252    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-envoy\") pod \"e21a973c-98db-465f-b02e-f8e961c420ec\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") "
Nov 03 16:39:25.355422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355295    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-socket\") pod \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") "
Nov 03 16:39:25.355422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355335    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-certs\") pod \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") "
Nov 03 16:39:25.355422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355371    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istiod-ca-cert" pod "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" (UID: "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:25.355422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355404    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e21a973c-98db-465f-b02e-f8e961c420ec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e21a973c-98db-465f-b02e-f8e961c420ec-istiod-ca-cert\") pod \"e21a973c-98db-465f-b02e-f8e961c420ec\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") "
Nov 03 16:39:25.355782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355462    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-podinfo\") pod \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") "
Nov 03 16:39:25.355782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355584    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-socket\") pod \"e21a973c-98db-465f-b02e-f8e961c420ec\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") "
Nov 03 16:39:25.355782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355684    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-certs\") pod \"e21a973c-98db-465f-b02e-f8e961c420ec\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") "
Nov 03 16:39:25.355782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355759    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-istio-token\" (UniqueName: \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-istio-token\") pod \"e21a973c-98db-465f-b02e-f8e961c420ec\" (UID: \"e21a973c-98db-465f-b02e-f8e961c420ec\") "
Nov 03 16:39:25.355975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355791    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-data\") pod \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") "
Nov 03 16:39:25.355975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355864    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-kube-api-access-cwwh8\" (UniqueName: \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-kube-api-access-cwwh8\") pod \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\" (UID: \"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8\") "
Nov 03 16:39:25.356712 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.356688    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.356867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.356716    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.356867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.355859    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-certs" pod "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" (UID: "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.356867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.356052    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-socket" pod "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" (UID: "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.356867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.356230    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-certs" pod "e21a973c-98db-465f-b02e-f8e961c420ec" (UID: "e21a973c-98db-465f-b02e-f8e961c420ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.356867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.356494    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e21a973c-98db-465f-b02e-f8e961c420ec-istiod-ca-cert" pod "e21a973c-98db-465f-b02e-f8e961c420ec" (UID: "e21a973c-98db-465f-b02e-f8e961c420ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:25.357110 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.357026    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-credential-socket" pod "e21a973c-98db-465f-b02e-f8e961c420ec" (UID: "e21a973c-98db-465f-b02e-f8e961c420ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.357236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.357220    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-credential-socket" pod "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" (UID: "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.359997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.359631    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-socket" pod "e21a973c-98db-465f-b02e-f8e961c420ec" (UID: "e21a973c-98db-465f-b02e-f8e961c420ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.359997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.359685    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-podinfo" pod "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" (UID: "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:25.359997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.359690    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e21a973c-98db-465f-b02e-f8e961c420ec-istio-podinfo" pod "e21a973c-98db-465f-b02e-f8e961c420ec" (UID: "e21a973c-98db-465f-b02e-f8e961c420ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:25.359997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.359756    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-token" pod "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" (UID: "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:25.359997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.359799    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"]
Nov 03 16:39:25.359997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.359820    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-envoy" pod "e21a973c-98db-465f-b02e-f8e961c420ec" (UID: "e21a973c-98db-465f-b02e-f8e961c420ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.359997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.359892    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-envoy" pod "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" (UID: "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.360618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.360587    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-data" pod "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" (UID: "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.361623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.361601    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-kube-api-access-z4mqd" pod "e21a973c-98db-465f-b02e-f8e961c420ec" (UID: "e21a973c-98db-465f-b02e-f8e961c420ec"). InnerVolumeSpecName "kube-api-access-z4mqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:25.362061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.362039    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-kube-api-access-cwwh8" pod "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" (UID: "fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8"). InnerVolumeSpecName "kube-api-access-cwwh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:25.363527 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.363505    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-istio-token" pod "e21a973c-98db-465f-b02e-f8e961c420ec" (UID: "e21a973c-98db-465f-b02e-f8e961c420ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:25.364109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.364091    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wccsg"]
Nov 03 16:39:25.368877 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.368857    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:39:25.457564 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.457535    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-kube-api-access-szjrn\" (UniqueName: \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-kube-api-access-szjrn\") pod \"50ad46f3-42de-4277-846a-8be1e943e2b9\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") "
Nov 03 16:39:25.457736 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.457609    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-podinfo\") pod \"50ad46f3-42de-4277-846a-8be1e943e2b9\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") "
Nov 03 16:39:25.457736 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.457662    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-data\") pod \"50ad46f3-42de-4277-846a-8be1e943e2b9\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") "
Nov 03 16:39:25.457838 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.457720    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-certs\") pod \"50ad46f3-42de-4277-846a-8be1e943e2b9\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") "
Nov 03 16:39:25.457838 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.457789    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-token\" (UniqueName: \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-token\") pod \"50ad46f3-42de-4277-846a-8be1e943e2b9\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") "
Nov 03 16:39:25.457911 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.457853    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-envoy\") pod \"50ad46f3-42de-4277-846a-8be1e943e2b9\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") "
Nov 03 16:39:25.457996 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.457977    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-credential-socket\") pod \"50ad46f3-42de-4277-846a-8be1e943e2b9\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") "
Nov 03 16:39:25.458059 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.458032    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-data" pod "50ad46f3-42de-4277-846a-8be1e943e2b9" (UID: "50ad46f3-42de-4277-846a-8be1e943e2b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.458119 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.458052    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-certs" pod "50ad46f3-42de-4277-846a-8be1e943e2b9" (UID: "50ad46f3-42de-4277-846a-8be1e943e2b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.458178 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.458162    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50ad46f3-42de-4277-846a-8be1e943e2b9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ad46f3-42de-4277-846a-8be1e943e2b9-istiod-ca-cert\") pod \"50ad46f3-42de-4277-846a-8be1e943e2b9\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") "
Nov 03 16:39:25.458231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.458218    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-socket\") pod \"50ad46f3-42de-4277-846a-8be1e943e2b9\" (UID: \"50ad46f3-42de-4277-846a-8be1e943e2b9\") "
Nov 03 16:39:25.458325 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.458299    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-credential-socket" pod "50ad46f3-42de-4277-846a-8be1e943e2b9" (UID: "50ad46f3-42de-4277-846a-8be1e943e2b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.458653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.458570    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-socket" pod "50ad46f3-42de-4277-846a-8be1e943e2b9" (UID: "50ad46f3-42de-4277-846a-8be1e943e2b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.458851 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.458756    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50ad46f3-42de-4277-846a-8be1e943e2b9-istiod-ca-cert" pod "50ad46f3-42de-4277-846a-8be1e943e2b9" (UID: "50ad46f3-42de-4277-846a-8be1e943e2b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:25.459465 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459446    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.459577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459556    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.459577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459574    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.459699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459588    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwwh8\" (UniqueName: \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-kube-api-access-cwwh8\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.459699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459604    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.459699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459618    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.459699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459631    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.459699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459645    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4mqd\" (UniqueName: \"kubernetes.io/projected/e21a973c-98db-465f-b02e-f8e961c420ec-kube-api-access-z4mqd\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.459699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459662    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.459699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459676    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459691    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e21a973c-98db-465f-b02e-f8e961c420ec-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459753    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459778    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459793    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459810    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459823    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459838    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e21a973c-98db-465f-b02e-f8e961c420ec-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459853    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ad46f3-42de-4277-846a-8be1e943e2b9-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459866    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459880    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e21a973c-98db-465f-b02e-f8e961c420ec-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460060 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.459894    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.460606 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.460576    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-envoy" pod "50ad46f3-42de-4277-846a-8be1e943e2b9" (UID: "50ad46f3-42de-4277-846a-8be1e943e2b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.460702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.460582    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-podinfo" pod "50ad46f3-42de-4277-846a-8be1e943e2b9" (UID: "50ad46f3-42de-4277-846a-8be1e943e2b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:25.460702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.460645    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-token" pod "50ad46f3-42de-4277-846a-8be1e943e2b9" (UID: "50ad46f3-42de-4277-846a-8be1e943e2b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:25.460908 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.460889    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-kube-api-access-szjrn" pod "50ad46f3-42de-4277-846a-8be1e943e2b9" (UID: "50ad46f3-42de-4277-846a-8be1e943e2b9"). InnerVolumeSpecName "kube-api-access-szjrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:25.467516 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.467495    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"]
Nov 03 16:39:25.476430 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.476407    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-6ae690-openshift-default-6799b84845-wxngm"]
Nov 03 16:39:25.560616 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.560585    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-envoy\") pod \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") "
Nov 03 16:39:25.560756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.560680    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istiod-ca-cert\") pod \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") "
Nov 03 16:39:25.560824 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.560765    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-kube-api-access-sl9t4\" (UniqueName: \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-kube-api-access-sl9t4\") pod \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") "
Nov 03 16:39:25.560824 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.560816    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-podinfo\") pod \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") "
Nov 03 16:39:25.560937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.560922    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-token\" (UniqueName: \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-token\") pod \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") "
Nov 03 16:39:25.561010 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.560990    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-data\") pod \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") "
Nov 03 16:39:25.561103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561070    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istiod-ca-cert" pod "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" (UID: "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:25.561160 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561107    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-credential-socket\") pod \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") "
Nov 03 16:39:25.561209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561178    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-certs\") pod \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") "
Nov 03 16:39:25.561260 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561215    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-socket\") pod \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\" (UID: \"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a\") "
Nov 03 16:39:25.561310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561256    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-data" pod "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" (UID: "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.561619 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561419    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-certs" pod "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" (UID: "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.561619 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561490    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-credential-socket" pod "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" (UID: "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.561619 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561526    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-socket" pod "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" (UID: "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.561933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561913    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.561994 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561936    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.561994 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561946    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.561994 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561959    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.561994 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561973    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.561994 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561982    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.561994 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.561991    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.562273 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.562000    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szjrn\" (UniqueName: \"kubernetes.io/projected/50ad46f3-42de-4277-846a-8be1e943e2b9-kube-api-access-szjrn\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.562273 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.562009    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ad46f3-42de-4277-846a-8be1e943e2b9-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.563227 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.563204    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-envoy" pod "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" (UID: "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:25.563417 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.563399    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-kube-api-access-sl9t4" pod "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" (UID: "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a"). InnerVolumeSpecName "kube-api-access-sl9t4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:25.563621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.563586    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-token" pod "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" (UID: "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:25.563621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.563587    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-podinfo" pod "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" (UID: "6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:25.664047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.664016    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.664047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.664043    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.664047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.664054    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl9t4\" (UniqueName: \"kubernetes.io/projected/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-kube-api-access-sl9t4\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.664272 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.664064    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:25.769241 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.769202    5605 generic.go:356] "Generic (PLEG): container finished" podID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerID="b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80" exitCode=0
Nov 03 16:39:25.769866 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.769265    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"
Nov 03 16:39:25.769866 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.769279    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" event={"ID":"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8","Type":"ContainerDied","Data":"b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80"}
Nov 03 16:39:25.769866 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.769338    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh" event={"ID":"fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8","Type":"ContainerDied","Data":"986dee5bb2babc20c85ad741ed4efc28249879ed4b758b57acfc045dfb3558ba"}
Nov 03 16:39:25.769866 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.769370    5605 scope.go:122] "RemoveContainer" containerID="b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80"
Nov 03 16:39:25.774480 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.773032    5605 generic.go:356] "Generic (PLEG): container finished" podID="e21a973c-98db-465f-b02e-f8e961c420ec" containerID="a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966" exitCode=0
Nov 03 16:39:25.774480 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.773116    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"
Nov 03 16:39:25.774480 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.773142    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" event={"ID":"e21a973c-98db-465f-b02e-f8e961c420ec","Type":"ContainerDied","Data":"a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966"}
Nov 03 16:39:25.774480 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.773203    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm" event={"ID":"e21a973c-98db-465f-b02e-f8e961c420ec","Type":"ContainerDied","Data":"ede00f6d9556ca4fcaaea2fc8ae1cecba1005b5677e95a8f0bd5151440a06ad0"}
Nov 03 16:39:25.783395 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.783370    5605 generic.go:356] "Generic (PLEG): container finished" podID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerID="223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4" exitCode=0
Nov 03 16:39:25.783515 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.783416    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"
Nov 03 16:39:25.783598 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.783448    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" event={"ID":"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a","Type":"ContainerDied","Data":"223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4"}
Nov 03 16:39:25.783649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.783611    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p" event={"ID":"6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a","Type":"ContainerDied","Data":"fe5678e72cf8cb1c2373c3e3861caa0eed405b2a91e18581d0d68540b069049c"}
Nov 03 16:39:25.787467 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.787444    5605 generic.go:356] "Generic (PLEG): container finished" podID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerID="f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b" exitCode=0
Nov 03 16:39:25.787551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.787526    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" event={"ID":"50ad46f3-42de-4277-846a-8be1e943e2b9","Type":"ContainerDied","Data":"f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b"}
Nov 03 16:39:25.787648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.787566    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" event={"ID":"50ad46f3-42de-4277-846a-8be1e943e2b9","Type":"ContainerDied","Data":"6eaa1d5a38427596565e08b2ba0912c50a70d5004be1c9869b16a6af1daf34bd"}
Nov 03 16:39:25.787648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.787628    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"
Nov 03 16:39:25.787906 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.787882    5605 scope.go:122] "RemoveContainer" containerID="b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80"
Nov 03 16:39:25.788478 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:25.788335    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80\": container with ID starting with b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80 not found: ID does not exist" containerID="b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80"
Nov 03 16:39:25.788478 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.788369    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80"} err="failed to get container status \"b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80\": rpc error: code = NotFound desc = could not find container \"b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80\": container with ID starting with b347187219bb2f7ead73e6fea40ba69aa3f6b949b341d39fab8ba32f14768e80 not found: ID does not exist"
Nov 03 16:39:25.788478 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.788392    5605 scope.go:122] "RemoveContainer" containerID="a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966"
Nov 03 16:39:25.799790 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.799775    5605 scope.go:122] "RemoveContainer" containerID="a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966"
Nov 03 16:39:25.800028 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:25.800011    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966\": container with ID starting with a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966 not found: ID does not exist" containerID="a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966"
Nov 03 16:39:25.800084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.800035    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966"} err="failed to get container status \"a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966\": rpc error: code = NotFound desc = could not find container \"a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966\": container with ID starting with a3c152860bfa557d633ca7c02648587802e6a1092c3a14be3d6ca7d7e99ab966 not found: ID does not exist"
Nov 03 16:39:25.800084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.800052    5605 scope.go:122] "RemoveContainer" containerID="223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4"
Nov 03 16:39:25.809799 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.809717    5605 scope.go:122] "RemoveContainer" containerID="8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5"
Nov 03 16:39:25.823289 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.823268    5605 scope.go:122] "RemoveContainer" containerID="223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4"
Nov 03 16:39:25.823638 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:25.823615    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4\": container with ID starting with 223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4 not found: ID does not exist" containerID="223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4"
Nov 03 16:39:25.823738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.823647    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4"} err="failed to get container status \"223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4\": rpc error: code = NotFound desc = could not find container \"223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4\": container with ID starting with 223a567163ba6672d29a5baa8ca6dce7820a43dcee20db92859208806cf892a4 not found: ID does not exist"
Nov 03 16:39:25.823738 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.823671    5605 scope.go:122] "RemoveContainer" containerID="8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5"
Nov 03 16:39:25.824004 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:25.823985    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5\": container with ID starting with 8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5 not found: ID does not exist" containerID="8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5"
Nov 03 16:39:25.824056 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.824014    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5"} err="failed to get container status \"8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5\": rpc error: code = NotFound desc = could not find container \"8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5\": container with ID starting with 8ea0621614e4de990b63c1f7c41870e0b993a80275e53a12f9a49161586adcb5 not found: ID does not exist"
Nov 03 16:39:25.824056 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.824032    5605 scope.go:122] "RemoveContainer" containerID="f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b"
Nov 03 16:39:25.836982 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.836960    5605 scope.go:122] "RemoveContainer" containerID="f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b"
Nov 03 16:39:25.837280 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:25.837261    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b\": container with ID starting with f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b not found: ID does not exist" containerID="f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b"
Nov 03 16:39:25.837325 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:25.837287    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b"} err="failed to get container status \"f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b\": rpc error: code = NotFound desc = could not find container \"f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b\": container with ID starting with f510df76acda7ecc963bde8beb43308f231e8dacd1ca8abf118d6e57d020898b not found: ID does not exist"
Nov 03 16:39:26.196525 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.196484    5605 patch_prober.go:29] interesting pod/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:39:26.196847 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.196815    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:39:26.262816 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.258205    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"]
Nov 03 16:39:26.308526 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.308472    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-m4cnm"]
Nov 03 16:39:26.372704 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.372671    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" path="/var/lib/kubelet/pods/231bee41-ee7a-4b55-ad4c-4b39a0104869/volumes"
Nov 03 16:39:26.373923 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.373882    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" path="/var/lib/kubelet/pods/806a66bb-48eb-41ae-8fbe-c7869b6040f5/volumes"
Nov 03 16:39:26.374672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.374654    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" path="/var/lib/kubelet/pods/8584968f-9bed-42ab-bf5c-9d238e2ca0cd/volumes"
Nov 03 16:39:26.375927 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.375908    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" path="/var/lib/kubelet/pods/a69b3be4-c69f-48e6-881a-436892b08ec3/volumes"
Nov 03 16:39:26.376689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.376672    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" path="/var/lib/kubelet/pods/e21a973c-98db-465f-b02e-f8e961c420ec/volumes"
Nov 03 16:39:26.437591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.437546    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"]
Nov 03 16:39:26.488805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.476883    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-4cplh"]
Nov 03 16:39:26.534101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.534070    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"]
Nov 03 16:39:26.570792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.564849    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-56m8p"]
Nov 03 16:39:26.623856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.623820    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"]
Nov 03 16:39:26.665749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:26.641345    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-bcd756-openshift-default-fb4b6cdf-t29t2"]
Nov 03 16:39:28.362757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:28.362383    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" path="/var/lib/kubelet/pods/50ad46f3-42de-4277-846a-8be1e943e2b9/volumes"
Nov 03 16:39:28.363415 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:28.363225    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" path="/var/lib/kubelet/pods/6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a/volumes"
Nov 03 16:39:28.367756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:28.363962    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" path="/var/lib/kubelet/pods/fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8/volumes"
Nov 03 16:39:29.262754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:29.257812    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:39:29.289749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:29.287808    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:39:30.642633 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:30.642597    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"]
Nov 03 16:39:30.643087 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:30.642866    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9" podUID="59ae65c7-7bbb-49ac-8876-db9f7d84031c" containerName="maas-api" containerID="cri-o://4d0d3a03f9d06a3142b56ea513c47580f632b3c92cbf471952a80eeaf4f6fc7f" gracePeriod=30
Nov 03 16:39:30.970241 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:30.970148    5605 generic.go:356] "Generic (PLEG): container finished" podID="59ae65c7-7bbb-49ac-8876-db9f7d84031c" containerID="4d0d3a03f9d06a3142b56ea513c47580f632b3c92cbf471952a80eeaf4f6fc7f" exitCode=0
Nov 03 16:39:30.970694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:30.970669    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9" event={"ID":"59ae65c7-7bbb-49ac-8876-db9f7d84031c","Type":"ContainerDied","Data":"4d0d3a03f9d06a3142b56ea513c47580f632b3c92cbf471952a80eeaf4f6fc7f"}
Nov 03 16:39:31.116720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.103185    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:39:31.224402 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.224318    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-metrics-certs\") pod \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") "
Nov 03 16:39:31.225183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.225152    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59ae65c7-7bbb-49ac-8876-db9f7d84031c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ae65c7-7bbb-49ac-8876-db9f7d84031c-openshift-service-ca\") pod \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") "
Nov 03 16:39:31.225670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.225642    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59ae65c7-7bbb-49ac-8876-db9f7d84031c-openshift-service-ca" pod "59ae65c7-7bbb-49ac-8876-db9f7d84031c" (UID: "59ae65c7-7bbb-49ac-8876-db9f7d84031c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:31.229931 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.229901    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-metrics-certs" pod "59ae65c7-7bbb-49ac-8876-db9f7d84031c" (UID: "59ae65c7-7bbb-49ac-8876-db9f7d84031c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:39:31.230411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.230389    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-maas-api-tls\") pod \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") "
Nov 03 16:39:31.230935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.230918    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59ae65c7-7bbb-49ac-8876-db9f7d84031c-kube-api-access-rq7xj\" (UniqueName: \"kubernetes.io/projected/59ae65c7-7bbb-49ac-8876-db9f7d84031c-kube-api-access-rq7xj\") pod \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\" (UID: \"59ae65c7-7bbb-49ac-8876-db9f7d84031c\") "
Nov 03 16:39:31.232074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.232055    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:31.232214 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.232199    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59ae65c7-7bbb-49ac-8876-db9f7d84031c-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:31.245191 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.244345    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-maas-api-tls" pod "59ae65c7-7bbb-49ac-8876-db9f7d84031c" (UID: "59ae65c7-7bbb-49ac-8876-db9f7d84031c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:39:31.246454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.246428    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ae65c7-7bbb-49ac-8876-db9f7d84031c-kube-api-access-rq7xj" pod "59ae65c7-7bbb-49ac-8876-db9f7d84031c" (UID: "59ae65c7-7bbb-49ac-8876-db9f7d84031c"). InnerVolumeSpecName "kube-api-access-rq7xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:31.333474 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.333443    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59ae65c7-7bbb-49ac-8876-db9f7d84031c-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:31.333474 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.333469    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq7xj\" (UniqueName: \"kubernetes.io/projected/59ae65c7-7bbb-49ac-8876-db9f7d84031c-kube-api-access-rq7xj\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:31.980429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.980388    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9" event={"ID":"59ae65c7-7bbb-49ac-8876-db9f7d84031c","Type":"ContainerDied","Data":"1fafeb71a4d6d139e7d8dc80ed862d7ebef60c9ce104f4aaec45c6ce92e23829"}
Nov 03 16:39:31.980908 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.980450    5605 scope.go:122] "RemoveContainer" containerID="4d0d3a03f9d06a3142b56ea513c47580f632b3c92cbf471952a80eeaf4f6fc7f"
Nov 03 16:39:31.980908 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:31.980485    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"
Nov 03 16:39:32.153401 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.153367    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"]
Nov 03 16:39:32.204636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.204604    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b70a8a53-c67bbb9d6-nv8z9"]
Nov 03 16:39:32.309748 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.309647    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59ae65c7-7bbb-49ac-8876-db9f7d84031c" path="/var/lib/kubelet/pods/59ae65c7-7bbb-49ac-8876-db9f7d84031c/volumes"
Nov 03 16:39:32.462996 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.462963    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"]
Nov 03 16:39:32.463628 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463613    5605 cpu_manager.go:414] "containerMap: removing container" podUID="896b4895-c8a7-4fb8-960d-22a0603a8a34" containerName="payload-processing"
Nov 03 16:39:32.463672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463630    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="896b4895-c8a7-4fb8-960d-22a0603a8a34" containerName="payload-processing"
Nov 03 16:39:32.463672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463643    5605 cpu_manager.go:414] "containerMap: removing container" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy"
Nov 03 16:39:32.463672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463648    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy"
Nov 03 16:39:32.463672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463661    5605 cpu_manager.go:414] "containerMap: removing container" podUID="11f0533a-48e2-49d7-8106-6d1c40baa827" containerName="payload-pre-processing"
Nov 03 16:39:32.463672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463669    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11f0533a-48e2-49d7-8106-6d1c40baa827" containerName="payload-pre-processing"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463680    5605 cpu_manager.go:414] "containerMap: removing container" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463686    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463699    5605 cpu_manager.go:414] "containerMap: removing container" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463704    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463712    5605 cpu_manager.go:414] "containerMap: removing container" podUID="59ae65c7-7bbb-49ac-8876-db9f7d84031c" containerName="maas-api"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463717    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59ae65c7-7bbb-49ac-8876-db9f7d84031c" containerName="maas-api"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463750    5605 cpu_manager.go:414] "containerMap: removing container" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463756    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463762    5605 cpu_manager.go:414] "containerMap: removing container" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463767    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463775    5605 cpu_manager.go:414] "containerMap: removing container" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463779    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463787    5605 cpu_manager.go:414] "containerMap: removing container" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463791    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463802    5605 cpu_manager.go:414] "containerMap: removing container" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463806    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463812    5605 cpu_manager.go:414] "containerMap: removing container" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463819    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463835    5605 cpu_manager.go:414] "containerMap: removing container" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy"
Nov 03 16:39:32.463880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463839    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463950    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463960    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="50ad46f3-42de-4277-846a-8be1e943e2b9" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463966    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="231bee41-ee7a-4b55-ad4c-4b39a0104869" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463973    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd9a9aa1-9efb-42aa-8b33-5ddecb40c9b8" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463981    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.463992    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="806a66bb-48eb-41ae-8fbe-c7869b6040f5" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.464001    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="a69b3be4-c69f-48e6-881a-436892b08ec3" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.464013    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="896b4895-c8a7-4fb8-960d-22a0603a8a34" containerName="payload-processing"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.464019    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="6940c25a-c0b2-4502-bdcf-1f8b32c0fd3a" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.464052    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="11f0533a-48e2-49d7-8106-6d1c40baa827" containerName="payload-pre-processing"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.464059    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="e21a973c-98db-465f-b02e-f8e961c420ec" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.464070    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="8584968f-9bed-42ab-bf5c-9d238e2ca0cd" containerName="istio-proxy"
Nov 03 16:39:32.464531 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.464080    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="59ae65c7-7bbb-49ac-8876-db9f7d84031c" containerName="maas-api"
Nov 03 16:39:32.469216 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.469196    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.525192 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.525158    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"]
Nov 03 16:39:32.525348 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.525219    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-d5937b-serving-cert\""
Nov 03 16:39:32.557607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.557578    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c18b4cbe-f908-454c-bf78-612a5b64ae00-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.557814 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.557617    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.557970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.557950    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz7zp\" (UniqueName: \"kubernetes.io/projected/c18b4cbe-f908-454c-bf78-612a5b64ae00-kube-api-access-lz7zp\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.558034 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.557996    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.659720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.659682    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz7zp\" (UniqueName: \"kubernetes.io/projected/c18b4cbe-f908-454c-bf78-612a5b64ae00-kube-api-access-lz7zp\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.659933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.659760    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.659933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.659877    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c18b4cbe-f908-454c-bf78-612a5b64ae00-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.659933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.659899    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.659933 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:32.659904    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-serving-cert: secret "maas-api-e2e-shared-a-w0-d5937b-serving-cert" not found
Nov 03 16:39:32.660139 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:32.659972    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-maas-api-tls podName:c18b4cbe-f908-454c-bf78-612a5b64ae00 nodeName:}" failed. No retries permitted until 2026-11-03 16:39:33.159950634 +0000 UTC m=+1658.984401901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-maas-api-tls") pod "maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8" (UID: "c18b4cbe-f908-454c-bf78-612a5b64ae00") : secret "maas-api-e2e-shared-a-w0-d5937b-serving-cert" not found
Nov 03 16:39:32.660508 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.660483    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c18b4cbe-f908-454c-bf78-612a5b64ae00-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.662399 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.662379    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:32.778694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:32.778612    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz7zp\" (UniqueName: \"kubernetes.io/projected/c18b4cbe-f908-454c-bf78-612a5b64ae00-kube-api-access-lz7zp\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:33.170692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:33.170651    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:33.173426 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:33.173391    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:33.382609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:33.382582    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:33.394870 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:33.394836    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:39:33.777825 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:33.777747    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"]
Nov 03 16:39:33.781403 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:39:33.781371    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc18b4cbe_f908_454c_bf78_612a5b64ae00.slice/crio-2a90b9ce7e4a992a50c28b843aa0b09e1b3f2bc8dbb4da2b448e2000985adb13 WatchSource:0}: Error finding container 2a90b9ce7e4a992a50c28b843aa0b09e1b3f2bc8dbb4da2b448e2000985adb13: Status 404 returned error can't find the container with id 2a90b9ce7e4a992a50c28b843aa0b09e1b3f2bc8dbb4da2b448e2000985adb13
Nov 03 16:39:34.009884 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:34.009846    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8" event={"ID":"c18b4cbe-f908-454c-bf78-612a5b64ae00","Type":"ContainerStarted","Data":"2a90b9ce7e4a992a50c28b843aa0b09e1b3f2bc8dbb4da2b448e2000985adb13"}
Nov 03 16:39:35.022762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:35.022698    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8" event={"ID":"c18b4cbe-f908-454c-bf78-612a5b64ae00","Type":"ContainerStarted","Data":"7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c"}
Nov 03 16:39:35.023524 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:35.023504    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:35.060470 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:35.060443    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"]
Nov 03 16:39:35.060641 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:35.060622    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy" containerID="cri-o://ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309" gracePeriod=30
Nov 03 16:39:35.182753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:35.179300    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8" podStartSLOduration=2.399211192 podStartE2EDuration="3.179276866s" totalImagesPullingTime="780.065674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:39:35.164209822 +0000 UTC m=+1660.988661149" watchObservedRunningTime="2026-11-03 16:39:35.179276866 +0000 UTC m=+1661.003728137"
Nov 03 16:39:37.118754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.102576    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"]
Nov 03 16:39:37.137147 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.137117    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.202764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.189238    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"]
Nov 03 16:39:37.227956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.227919    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-credential-socket\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.228181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.228162    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-token\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.228292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.228276    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-socket\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.228433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.228416    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/423c492d-e0e6-442f-949d-c4cee39a9d04-istiod-ca-cert\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.228598 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.228582    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-data\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.228836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.228814    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stwl9\" (UniqueName: \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-kube-api-access-stwl9\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.229029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.229012    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-certs\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.229189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.229174    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-envoy\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.229660 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.229636    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-podinfo\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.251256 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.249876    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-cc4dcd-openshift-default-dockercfg-ms6l9\""
Nov 03 16:39:37.441237 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.441151    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-socket\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.443623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.443594    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/423c492d-e0e6-442f-949d-c4cee39a9d04-istiod-ca-cert\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.444831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.444781    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-data\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.445903 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.445881    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stwl9\" (UniqueName: \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-kube-api-access-stwl9\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.446461 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.446439    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-certs\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.446739 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.446696    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-envoy\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.447209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.447170    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-podinfo\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.447535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.447513    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-credential-socket\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.449197 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.449180    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-token\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.449869 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.448186    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-certs\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.450186 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.445719    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-data\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.450186 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.443456    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-socket\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.450186 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.449116    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-credential-socket\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.450186 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.444494    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/423c492d-e0e6-442f-949d-c4cee39a9d04-istiod-ca-cert\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.467210 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.467052    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-podinfo\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.472813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.471092    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-envoy\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.510091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.510061    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stwl9\" (UniqueName: \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-kube-api-access-stwl9\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.514342 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.514313    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-token\") pod \"e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") " pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:37.557977 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.557933    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:38.000604 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:37.996579    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"]
Nov 03 16:39:38.002653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:38.002614    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:39:38.002747 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:38.002693    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:39:38.002747 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:38.002743    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:39:38.085353 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:38.085326    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" event={"ID":"423c492d-e0e6-442f-949d-c4cee39a9d04","Type":"ContainerStarted","Data":"2d24653961b571d899c4325e37a6b86d1d412864d6fbb8d867e430d3e5b20b27"}
Nov 03 16:39:39.116679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:39.114270    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" event={"ID":"423c492d-e0e6-442f-949d-c4cee39a9d04","Type":"ContainerStarted","Data":"fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8"}
Nov 03 16:39:39.293245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:39.293194    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" podStartSLOduration=2.2931783980000002 podStartE2EDuration="2.293178398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:39:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:39:39.290170821 +0000 UTC m=+1665.114622092" watchObservedRunningTime="2026-11-03 16:39:39.293178398 +0000 UTC m=+1665.117629669"
Nov 03 16:39:39.558128 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:39.558034    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:39.563773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:39.563748    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:40.121237 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.121202    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:40.122084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.122058    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:39:40.375236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.375208    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:39:40.399944 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.399910    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-kube-api-access-dmv64\" (UniqueName: \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-kube-api-access-dmv64\") pod \"a8c082b7-7db2-4f35-a62d-51750ca323ab\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") "
Nov 03 16:39:40.400118 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400084    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-token\" (UniqueName: \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-token\") pod \"a8c082b7-7db2-4f35-a62d-51750ca323ab\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") "
Nov 03 16:39:40.400159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400147    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-certs\") pod \"a8c082b7-7db2-4f35-a62d-51750ca323ab\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") "
Nov 03 16:39:40.400223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400207    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-credential-socket\") pod \"a8c082b7-7db2-4f35-a62d-51750ca323ab\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") "
Nov 03 16:39:40.400270 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400242    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-socket\") pod \"a8c082b7-7db2-4f35-a62d-51750ca323ab\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") "
Nov 03 16:39:40.400309 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400289    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-podinfo\") pod \"a8c082b7-7db2-4f35-a62d-51750ca323ab\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") "
Nov 03 16:39:40.400392 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400372    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8c082b7-7db2-4f35-a62d-51750ca323ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8c082b7-7db2-4f35-a62d-51750ca323ab-istiod-ca-cert\") pod \"a8c082b7-7db2-4f35-a62d-51750ca323ab\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") "
Nov 03 16:39:40.400494 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400476    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-envoy\") pod \"a8c082b7-7db2-4f35-a62d-51750ca323ab\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") "
Nov 03 16:39:40.400575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400493    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-credential-socket" pod "a8c082b7-7db2-4f35-a62d-51750ca323ab" (UID: "a8c082b7-7db2-4f35-a62d-51750ca323ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:40.400575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400504    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-certs" pod "a8c082b7-7db2-4f35-a62d-51750ca323ab" (UID: "a8c082b7-7db2-4f35-a62d-51750ca323ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:40.400575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400512    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-socket" pod "a8c082b7-7db2-4f35-a62d-51750ca323ab" (UID: "a8c082b7-7db2-4f35-a62d-51750ca323ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:40.400575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400559    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-data\") pod \"a8c082b7-7db2-4f35-a62d-51750ca323ab\" (UID: \"a8c082b7-7db2-4f35-a62d-51750ca323ab\") "
Nov 03 16:39:40.400891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400862    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-data" pod "a8c082b7-7db2-4f35-a62d-51750ca323ab" (UID: "a8c082b7-7db2-4f35-a62d-51750ca323ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:40.400891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.400881    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8c082b7-7db2-4f35-a62d-51750ca323ab-istiod-ca-cert" pod "a8c082b7-7db2-4f35-a62d-51750ca323ab" (UID: "a8c082b7-7db2-4f35-a62d-51750ca323ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:40.401571 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.401549    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:40.401675 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.401577    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:40.401675 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.401593    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:40.401675 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.401607    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8c082b7-7db2-4f35-a62d-51750ca323ab-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:40.401675 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.401621    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:40.403130 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.403110    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-kube-api-access-dmv64" pod "a8c082b7-7db2-4f35-a62d-51750ca323ab" (UID: "a8c082b7-7db2-4f35-a62d-51750ca323ab"). InnerVolumeSpecName "kube-api-access-dmv64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:40.403130 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.403124    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-token" pod "a8c082b7-7db2-4f35-a62d-51750ca323ab" (UID: "a8c082b7-7db2-4f35-a62d-51750ca323ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:40.403419 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.403397    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-podinfo" pod "a8c082b7-7db2-4f35-a62d-51750ca323ab" (UID: "a8c082b7-7db2-4f35-a62d-51750ca323ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:40.403504 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.403474    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-envoy" pod "a8c082b7-7db2-4f35-a62d-51750ca323ab" (UID: "a8c082b7-7db2-4f35-a62d-51750ca323ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:40.503591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.503550    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmv64\" (UniqueName: \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-kube-api-access-dmv64\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:40.503591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.503594    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:40.503824 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.503612    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:40.503824 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:40.503626    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8c082b7-7db2-4f35-a62d-51750ca323ab-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:41.128686 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.128638    5605 generic.go:356] "Generic (PLEG): container finished" podID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerID="ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309" exitCode=0
Nov 03 16:39:41.129116 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.128694    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"
Nov 03 16:39:41.129116 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.128755    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" event={"ID":"a8c082b7-7db2-4f35-a62d-51750ca323ab","Type":"ContainerDied","Data":"ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309"}
Nov 03 16:39:41.129116 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.128812    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv" event={"ID":"a8c082b7-7db2-4f35-a62d-51750ca323ab","Type":"ContainerDied","Data":"63ffce6e9f206305364a1c2f132f1d866d6c51c25007028fad97deb693c60bc5"}
Nov 03 16:39:41.129116 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.128838    5605 scope.go:122] "RemoveContainer" containerID="ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309"
Nov 03 16:39:41.139908 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.139890    5605 scope.go:122] "RemoveContainer" containerID="3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e"
Nov 03 16:39:41.149292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.149276    5605 scope.go:122] "RemoveContainer" containerID="ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309"
Nov 03 16:39:41.149523 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:41.149505    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309\": container with ID starting with ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309 not found: ID does not exist" containerID="ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309"
Nov 03 16:39:41.149582 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.149530    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309"} err="failed to get container status \"ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309\": rpc error: code = NotFound desc = could not find container \"ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309\": container with ID starting with ac85b56b86ce3088f4ea7cd326b04e8c2ffcb2e7c7a8e474f2a455ba2d493309 not found: ID does not exist"
Nov 03 16:39:41.149582 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.149550    5605 scope.go:122] "RemoveContainer" containerID="3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e"
Nov 03 16:39:41.149800 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:41.149782    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e\": container with ID starting with 3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e not found: ID does not exist" containerID="3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e"
Nov 03 16:39:41.149845 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.149804    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e"} err="failed to get container status \"3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e\": rpc error: code = NotFound desc = could not find container \"3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e\": container with ID starting with 3a8f0fe6554b538451ee834cf92d9119d050a8b6ac2e6aebc0dac2d21168b60e not found: ID does not exist"
Nov 03 16:39:41.354527 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.354498    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"]
Nov 03 16:39:41.409069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:41.407182    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b70a8a53-openshift-default-7fbbbcdd6c-jw5gv"]
Nov 03 16:39:42.083086 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:42.083047    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 16:39:42.312771 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:42.312693    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" path="/var/lib/kubelet/pods/a8c082b7-7db2-4f35-a62d-51750ca323ab/volumes"
Nov 03 16:39:43.165836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.165803    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"]
Nov 03 16:39:43.166045 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.166021    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy" containerID="cri-o://03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7" gracePeriod=30
Nov 03 16:39:43.217429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.217392    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"]
Nov 03 16:39:43.217662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.217634    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy" containerID="cri-o://ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4" gracePeriod=30
Nov 03 16:39:43.220766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.220739    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"]
Nov 03 16:39:43.220984 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.220947    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy" containerID="cri-o://8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc" gracePeriod=30
Nov 03 16:39:43.270094 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.270059    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"]
Nov 03 16:39:43.270325 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.270300    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy" containerID="cri-o://7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca" gracePeriod=30
Nov 03 16:39:43.274585 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.274558    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"]
Nov 03 16:39:43.275543 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.275521    5605 cpu_manager.go:414] "containerMap: removing container" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy"
Nov 03 16:39:43.275603 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.275547    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy"
Nov 03 16:39:43.275603 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.275599    5605 cpu_manager.go:414] "containerMap: removing container" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy"
Nov 03 16:39:43.275684 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.275607    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy"
Nov 03 16:39:43.275798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.275783    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy"
Nov 03 16:39:43.275851 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.275809    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8c082b7-7db2-4f35-a62d-51750ca323ab" containerName="istio-proxy"
Nov 03 16:39:43.281381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.281359    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.324236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.324202    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"]
Nov 03 16:39:43.360871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.360838    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02b89011-2c14-4641-86dd-6f05a6df2c33-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.361052 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.360958    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2lfg\" (UniqueName: \"kubernetes.io/projected/02b89011-2c14-4641-86dd-6f05a6df2c33-kube-api-access-s2lfg\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.361121 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.361052    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.361324 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.361298    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.391282 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.391254    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-438248-serving-cert\""
Nov 03 16:39:43.480801 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.476876    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02b89011-2c14-4641-86dd-6f05a6df2c33-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.480801 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.477316    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2lfg\" (UniqueName: \"kubernetes.io/projected/02b89011-2c14-4641-86dd-6f05a6df2c33-kube-api-access-s2lfg\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.480801 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.477488    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.480801 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.480404    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02b89011-2c14-4641-86dd-6f05a6df2c33-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.490688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.485370    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.490688 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:43.485861    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-serving-cert: secret "maas-api-e2e-shared-a-w1-438248-serving-cert" not found
Nov 03 16:39:43.490688 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:43.485965    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-maas-api-tls podName:02b89011-2c14-4641-86dd-6f05a6df2c33 nodeName:}" failed. No retries permitted until 2026-11-03 16:39:43.985940531 +0000 UTC m=+1669.810391781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-maas-api-tls") pod "maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm" (UID: "02b89011-2c14-4641-86dd-6f05a6df2c33") : secret "maas-api-e2e-shared-a-w1-438248-serving-cert" not found
Nov 03 16:39:43.490688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.488350    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.540254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.540223    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2lfg\" (UniqueName: \"kubernetes.io/projected/02b89011-2c14-4641-86dd-6f05a6df2c33-kube-api-access-s2lfg\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.993824 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.993784    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:43.996590 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:43.996551    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:44.228309 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:44.228277    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:44.616607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:44.616583    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"]
Nov 03 16:39:44.618465 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:39:44.618438    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02b89011_2c14_4641_86dd_6f05a6df2c33.slice/crio-2274caab63e6e4403a83daf8292f0275413ce37670285186967a8eba1a500a30 WatchSource:0}: Error finding container 2274caab63e6e4403a83daf8292f0275413ce37670285186967a8eba1a500a30: Status 404 returned error can't find the container with id 2274caab63e6e4403a83daf8292f0275413ce37670285186967a8eba1a500a30
Nov 03 16:39:45.156434 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:45.156401    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm" event={"ID":"02b89011-2c14-4641-86dd-6f05a6df2c33","Type":"ContainerStarted","Data":"2274caab63e6e4403a83daf8292f0275413ce37670285186967a8eba1a500a30"}
Nov 03 16:39:45.845440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:45.845405    5605 patch_prober.go:29] interesting pod/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:39:45.845861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:45.845462    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:39:46.164357 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:46.164317    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm" event={"ID":"02b89011-2c14-4641-86dd-6f05a6df2c33","Type":"ContainerStarted","Data":"8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce"}
Nov 03 16:39:46.164497 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:46.164413    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:46.307047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:46.306982    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm" podStartSLOduration=2.558705849 podStartE2EDuration="3.306956778s" totalImagesPullingTime="748.250929ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:39:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:39:46.303328397 +0000 UTC m=+1672.127779670" watchObservedRunningTime="2026-11-03 16:39:46.306956778 +0000 UTC m=+1672.131408126"
Nov 03 16:39:48.455773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.455646    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"]
Nov 03 16:39:48.456282 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.455991    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy" containerID="cri-o://1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9" gracePeriod=30
Nov 03 16:39:48.534236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.534205    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:39:48.645765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.644908    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:39:48.651689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.651603    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:39:48.678419 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.678393    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:39:48.693715 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.693686    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-kube-api-access-shph5\" (UniqueName: \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-kube-api-access-shph5\") pod \"6dce1a44-2d2b-488a-ac77-ff556bf12027\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") "
Nov 03 16:39:48.693858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.693812    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6dce1a44-2d2b-488a-ac77-ff556bf12027-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dce1a44-2d2b-488a-ac77-ff556bf12027-istiod-ca-cert\") pod \"6dce1a44-2d2b-488a-ac77-ff556bf12027\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") "
Nov 03 16:39:48.693858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.693839    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-data\") pod \"6dce1a44-2d2b-488a-ac77-ff556bf12027\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") "
Nov 03 16:39:48.693940 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.693864    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-envoy\") pod \"6dce1a44-2d2b-488a-ac77-ff556bf12027\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") "
Nov 03 16:39:48.693940 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.693895    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-certs\") pod \"6dce1a44-2d2b-488a-ac77-ff556bf12027\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") "
Nov 03 16:39:48.694011 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.693999    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-podinfo\") pod \"6dce1a44-2d2b-488a-ac77-ff556bf12027\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") "
Nov 03 16:39:48.694581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.694103    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-token\" (UniqueName: \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-token\") pod \"6dce1a44-2d2b-488a-ac77-ff556bf12027\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") "
Nov 03 16:39:48.694581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.694163    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-data" pod "6dce1a44-2d2b-488a-ac77-ff556bf12027" (UID: "6dce1a44-2d2b-488a-ac77-ff556bf12027"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.694581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.694199    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-certs" pod "6dce1a44-2d2b-488a-ac77-ff556bf12027" (UID: "6dce1a44-2d2b-488a-ac77-ff556bf12027"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.694581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.694233    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-credential-socket\") pod \"6dce1a44-2d2b-488a-ac77-ff556bf12027\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") "
Nov 03 16:39:48.694581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.694240    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dce1a44-2d2b-488a-ac77-ff556bf12027-istiod-ca-cert" pod "6dce1a44-2d2b-488a-ac77-ff556bf12027" (UID: "6dce1a44-2d2b-488a-ac77-ff556bf12027"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:48.694581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.694276    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-socket\") pod \"6dce1a44-2d2b-488a-ac77-ff556bf12027\" (UID: \"6dce1a44-2d2b-488a-ac77-ff556bf12027\") "
Nov 03 16:39:48.694581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.694530    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-socket" pod "6dce1a44-2d2b-488a-ac77-ff556bf12027" (UID: "6dce1a44-2d2b-488a-ac77-ff556bf12027"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.694581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.694494    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-credential-socket" pod "6dce1a44-2d2b-488a-ac77-ff556bf12027" (UID: "6dce1a44-2d2b-488a-ac77-ff556bf12027"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.695208 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.695184    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.695275 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.695213    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.695275 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.695228    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dce1a44-2d2b-488a-ac77-ff556bf12027-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.695275 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.695243    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.695275 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.695256    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.696213 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.696190    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-envoy" pod "6dce1a44-2d2b-488a-ac77-ff556bf12027" (UID: "6dce1a44-2d2b-488a-ac77-ff556bf12027"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.696292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.696221    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-kube-api-access-shph5" pod "6dce1a44-2d2b-488a-ac77-ff556bf12027" (UID: "6dce1a44-2d2b-488a-ac77-ff556bf12027"). InnerVolumeSpecName "kube-api-access-shph5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:48.696355 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.696341    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-podinfo" pod "6dce1a44-2d2b-488a-ac77-ff556bf12027" (UID: "6dce1a44-2d2b-488a-ac77-ff556bf12027"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:48.696435 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.696415    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-token" pod "6dce1a44-2d2b-488a-ac77-ff556bf12027" (UID: "6dce1a44-2d2b-488a-ac77-ff556bf12027"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:48.795987 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.795900    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-kube-api-access-dzbck\" (UniqueName: \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-kube-api-access-dzbck\") pod \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") "
Nov 03 16:39:48.795987 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.795983    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-podinfo\") pod \"5c153a4f-9b14-4934-84a8-ba29826e20dd\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") "
Nov 03 16:39:48.796183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796027    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-kube-api-access-2kq5f\" (UniqueName: \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-kube-api-access-2kq5f\") pod \"5c153a4f-9b14-4934-84a8-ba29826e20dd\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") "
Nov 03 16:39:48.796183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796054    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istiod-ca-cert\") pod \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") "
Nov 03 16:39:48.796183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796168    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c153a4f-9b14-4934-84a8-ba29826e20dd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c153a4f-9b14-4934-84a8-ba29826e20dd-istiod-ca-cert\") pod \"5c153a4f-9b14-4934-84a8-ba29826e20dd\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") "
Nov 03 16:39:48.796334 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796256    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-credential-socket\") pod \"5c153a4f-9b14-4934-84a8-ba29826e20dd\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") "
Nov 03 16:39:48.796334 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796301    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-podinfo\") pod \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") "
Nov 03 16:39:48.796433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796364    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-socket\") pod \"5c153a4f-9b14-4934-84a8-ba29826e20dd\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") "
Nov 03 16:39:48.796433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796399    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-certs\") pod \"ab094755-d4d2-40bc-a411-e73e6fd3372f\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") "
Nov 03 16:39:48.796544 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796430    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-envoy\") pod \"5c153a4f-9b14-4934-84a8-ba29826e20dd\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") "
Nov 03 16:39:48.796544 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796469    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-token\" (UniqueName: \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-token\") pod \"5c153a4f-9b14-4934-84a8-ba29826e20dd\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") "
Nov 03 16:39:48.796544 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796480    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istiod-ca-cert" pod "5e21fcd1-f5a4-46cf-a8da-c340063f7727" (UID: "5e21fcd1-f5a4-46cf-a8da-c340063f7727"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:48.796689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796547    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-podinfo\") pod \"ab094755-d4d2-40bc-a411-e73e6fd3372f\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") "
Nov 03 16:39:48.796689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796547    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c153a4f-9b14-4934-84a8-ba29826e20dd-istiod-ca-cert" pod "5c153a4f-9b14-4934-84a8-ba29826e20dd" (UID: "5c153a4f-9b14-4934-84a8-ba29826e20dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:48.796881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796620    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-data\") pod \"5c153a4f-9b14-4934-84a8-ba29826e20dd\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") "
Nov 03 16:39:48.796881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796757    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-certs" pod "ab094755-d4d2-40bc-a411-e73e6fd3372f" (UID: "ab094755-d4d2-40bc-a411-e73e6fd3372f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.796881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796791    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-envoy\") pod \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") "
Nov 03 16:39:48.796881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796828    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-token\" (UniqueName: \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-token\") pod \"ab094755-d4d2-40bc-a411-e73e6fd3372f\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796887    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-data\") pod \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796927    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-data\") pod \"ab094755-d4d2-40bc-a411-e73e6fd3372f\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796952    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-socket" pod "5c153a4f-9b14-4934-84a8-ba29826e20dd" (UID: "5c153a4f-9b14-4934-84a8-ba29826e20dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796973    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-credential-socket\") pod \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797002    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-socket\") pod \"ab094755-d4d2-40bc-a411-e73e6fd3372f\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797056    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-kube-api-access-7mklg\" (UniqueName: \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-kube-api-access-7mklg\") pod \"ab094755-d4d2-40bc-a411-e73e6fd3372f\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.796995    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-credential-socket" pod "5c153a4f-9b14-4934-84a8-ba29826e20dd" (UID: "5c153a4f-9b14-4934-84a8-ba29826e20dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797100    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-socket\") pod \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797128    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-credential-socket\") pod \"ab094755-d4d2-40bc-a411-e73e6fd3372f\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797182    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-token\" (UniqueName: \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-token\") pod \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797220    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-certs\") pod \"5c153a4f-9b14-4934-84a8-ba29826e20dd\" (UID: \"5c153a4f-9b14-4934-84a8-ba29826e20dd\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797232    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-data" pod "5c153a4f-9b14-4934-84a8-ba29826e20dd" (UID: "5c153a4f-9b14-4934-84a8-ba29826e20dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797285    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-certs\") pod \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\" (UID: \"5e21fcd1-f5a4-46cf-a8da-c340063f7727\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797311    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab094755-d4d2-40bc-a411-e73e6fd3372f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab094755-d4d2-40bc-a411-e73e6fd3372f-istiod-ca-cert\") pod \"ab094755-d4d2-40bc-a411-e73e6fd3372f\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797349    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-envoy\") pod \"ab094755-d4d2-40bc-a411-e73e6fd3372f\" (UID: \"ab094755-d4d2-40bc-a411-e73e6fd3372f\") "
Nov 03 16:39:48.797458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797372    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-data" pod "ab094755-d4d2-40bc-a411-e73e6fd3372f" (UID: "ab094755-d4d2-40bc-a411-e73e6fd3372f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.798226 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797471    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-credential-socket" pod "ab094755-d4d2-40bc-a411-e73e6fd3372f" (UID: "ab094755-d4d2-40bc-a411-e73e6fd3372f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.798226 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.797842    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-socket" pod "5e21fcd1-f5a4-46cf-a8da-c340063f7727" (UID: "5e21fcd1-f5a4-46cf-a8da-c340063f7727"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.798334 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798310    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798334    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798351    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shph5\" (UniqueName: \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-kube-api-access-shph5\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798367    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798382    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798396    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798410    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798424    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798438    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798451    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dce1a44-2d2b-488a-ac77-ff556bf12027-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798465    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798478    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c153a4f-9b14-4934-84a8-ba29826e20dd-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.798555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798491    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.799680 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.798716    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-certs" pod "5c153a4f-9b14-4934-84a8-ba29826e20dd" (UID: "5c153a4f-9b14-4934-84a8-ba29826e20dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.799680 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.799559    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-credential-socket" pod "5e21fcd1-f5a4-46cf-a8da-c340063f7727" (UID: "5e21fcd1-f5a4-46cf-a8da-c340063f7727"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.799680 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.799664    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-kube-api-access-2kq5f" pod "5c153a4f-9b14-4934-84a8-ba29826e20dd" (UID: "5c153a4f-9b14-4934-84a8-ba29826e20dd"). InnerVolumeSpecName "kube-api-access-2kq5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:48.800251 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.799900    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-certs" pod "5e21fcd1-f5a4-46cf-a8da-c340063f7727" (UID: "5e21fcd1-f5a4-46cf-a8da-c340063f7727"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.800251 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.800102    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-socket" pod "ab094755-d4d2-40bc-a411-e73e6fd3372f" (UID: "ab094755-d4d2-40bc-a411-e73e6fd3372f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.800411 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.800379    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab094755-d4d2-40bc-a411-e73e6fd3372f-istiod-ca-cert" pod "ab094755-d4d2-40bc-a411-e73e6fd3372f" (UID: "ab094755-d4d2-40bc-a411-e73e6fd3372f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:48.800466 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.800417    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-kube-api-access-dzbck" pod "5e21fcd1-f5a4-46cf-a8da-c340063f7727" (UID: "5e21fcd1-f5a4-46cf-a8da-c340063f7727"). InnerVolumeSpecName "kube-api-access-dzbck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:48.800466 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.800429    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-data" pod "5e21fcd1-f5a4-46cf-a8da-c340063f7727" (UID: "5e21fcd1-f5a4-46cf-a8da-c340063f7727"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.800633 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.800607    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-envoy" pod "ab094755-d4d2-40bc-a411-e73e6fd3372f" (UID: "ab094755-d4d2-40bc-a411-e73e6fd3372f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.801108 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.801079    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-podinfo" pod "ab094755-d4d2-40bc-a411-e73e6fd3372f" (UID: "ab094755-d4d2-40bc-a411-e73e6fd3372f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:48.801198 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.801155    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-envoy" pod "5c153a4f-9b14-4934-84a8-ba29826e20dd" (UID: "5c153a4f-9b14-4934-84a8-ba29826e20dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.801273 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.801245    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-token" pod "ab094755-d4d2-40bc-a411-e73e6fd3372f" (UID: "ab094755-d4d2-40bc-a411-e73e6fd3372f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:48.801542 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.801517    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-podinfo" pod "5e21fcd1-f5a4-46cf-a8da-c340063f7727" (UID: "5e21fcd1-f5a4-46cf-a8da-c340063f7727"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:48.801983 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.801953    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-token" pod "5c153a4f-9b14-4934-84a8-ba29826e20dd" (UID: "5c153a4f-9b14-4934-84a8-ba29826e20dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:48.802165 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.802142    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-kube-api-access-7mklg" pod "ab094755-d4d2-40bc-a411-e73e6fd3372f" (UID: "ab094755-d4d2-40bc-a411-e73e6fd3372f"). InnerVolumeSpecName "kube-api-access-7mklg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:48.802925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.802907    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-token" pod "5e21fcd1-f5a4-46cf-a8da-c340063f7727" (UID: "5e21fcd1-f5a4-46cf-a8da-c340063f7727"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:48.802925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.802916    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-podinfo" pod "5c153a4f-9b14-4934-84a8-ba29826e20dd" (UID: "5c153a4f-9b14-4934-84a8-ba29826e20dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:48.803281 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.803263    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-envoy" pod "5e21fcd1-f5a4-46cf-a8da-c340063f7727" (UID: "5e21fcd1-f5a4-46cf-a8da-c340063f7727"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:48.900594 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900556    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900594 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900597    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900615    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900631    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mklg\" (UniqueName: \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-kube-api-access-7mklg\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900646    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900660    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900673    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900687    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab094755-d4d2-40bc-a411-e73e6fd3372f-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900701    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900714    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzbck\" (UniqueName: \"kubernetes.io/projected/5e21fcd1-f5a4-46cf-a8da-c340063f7727-kube-api-access-dzbck\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900749    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900763    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kq5f\" (UniqueName: \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-kube-api-access-2kq5f\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900778    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900791    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.900811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900805    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c153a4f-9b14-4934-84a8-ba29826e20dd-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.901191 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900818    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.901191 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900832    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e21fcd1-f5a4-46cf-a8da-c340063f7727-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.901191 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.900844    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab094755-d4d2-40bc-a411-e73e6fd3372f-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:48.952788 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.952720    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"]
Nov 03 16:39:48.953035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.953013    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy" containerID="cri-o://5e6c091adb23699d176761af10ad19a254dfa027bb75aef25e85e6460e73cf46" gracePeriod=30
Nov 03 16:39:48.959228 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.958195    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"]
Nov 03 16:39:48.959228 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.958406    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy" containerID="cri-o://7554893895a7c1e24e2bba1aff977813e01e0bddb92bc4da89e6440a7f1fadd8" gracePeriod=30
Nov 03 16:39:48.965967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.965907    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"]
Nov 03 16:39:48.966669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.966569    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy" containerID="cri-o://bd57e88bbcb3fa578ac45961573ae740901070347ca9861f4730f0fcfe6ac73c" gracePeriod=30
Nov 03 16:39:48.974742 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.974234    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"]
Nov 03 16:39:48.974742 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:48.974507    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy" containerID="cri-o://85807ecc93933e92dbdffdbac1b60541f94b80bd121599bdfabb11c715dd4615" gracePeriod=30
Nov 03 16:39:49.188248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.188218    5605 generic.go:356] "Generic (PLEG): container finished" podID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerID="ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4" exitCode=0
Nov 03 16:39:49.188424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.188268    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"
Nov 03 16:39:49.188424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.188300    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" event={"ID":"5e21fcd1-f5a4-46cf-a8da-c340063f7727","Type":"ContainerDied","Data":"ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4"}
Nov 03 16:39:49.188424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.188360    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr" event={"ID":"5e21fcd1-f5a4-46cf-a8da-c340063f7727","Type":"ContainerDied","Data":"5e911b5a591bb4ffddf91c4c0d8b99cf305271f33b18aa0134337c41361e93fb"}
Nov 03 16:39:49.188424 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.188389    5605 scope.go:122] "RemoveContainer" containerID="ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4"
Nov 03 16:39:49.190092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.190068    5605 generic.go:356] "Generic (PLEG): container finished" podID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerID="7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca" exitCode=0
Nov 03 16:39:49.190225 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.190124    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" event={"ID":"ab094755-d4d2-40bc-a411-e73e6fd3372f","Type":"ContainerDied","Data":"7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca"}
Nov 03 16:39:49.190225 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.190188    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp" event={"ID":"ab094755-d4d2-40bc-a411-e73e6fd3372f","Type":"ContainerDied","Data":"50ad0bf23ef962e0100a3b2c68e465483135bbd888e6e0f9ee39dfa478137eb6"}
Nov 03 16:39:49.190420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.190395    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"
Nov 03 16:39:49.191951 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.191926    5605 generic.go:356] "Generic (PLEG): container finished" podID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerID="8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc" exitCode=0
Nov 03 16:39:49.192075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.191978    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"
Nov 03 16:39:49.192075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.192043    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" event={"ID":"5c153a4f-9b14-4934-84a8-ba29826e20dd","Type":"ContainerDied","Data":"8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc"}
Nov 03 16:39:49.192193 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.192113    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g" event={"ID":"5c153a4f-9b14-4934-84a8-ba29826e20dd","Type":"ContainerDied","Data":"43ff7a8d12ba83ae393eaec3e2d9e8e1d56003d63a5ae202a3d3aed8cb9c81d6"}
Nov 03 16:39:49.193688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.193665    5605 generic.go:356] "Generic (PLEG): container finished" podID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerID="03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7" exitCode=0
Nov 03 16:39:49.193819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.193754    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" event={"ID":"6dce1a44-2d2b-488a-ac77-ff556bf12027","Type":"ContainerDied","Data":"03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7"}
Nov 03 16:39:49.193819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.193790    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn" event={"ID":"6dce1a44-2d2b-488a-ac77-ff556bf12027","Type":"ContainerDied","Data":"c4fa982b9ca3855566cc004f30614da2d1d72d0ef0d80aed71e781420ac4c609"}
Nov 03 16:39:49.193940 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.193845    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"
Nov 03 16:39:49.224185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.224165    5605 scope.go:122] "RemoveContainer" containerID="ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4"
Nov 03 16:39:49.224558 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:49.224540    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4\": container with ID starting with ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4 not found: ID does not exist" containerID="ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4"
Nov 03 16:39:49.224621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.224566    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4"} err="failed to get container status \"ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4\": rpc error: code = NotFound desc = could not find container \"ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4\": container with ID starting with ad6d0ef1bb2ea1caf1f62064e036755d0cb96dec7822f38a3600be063a011cf4 not found: ID does not exist"
Nov 03 16:39:49.224621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.224584    5605 scope.go:122] "RemoveContainer" containerID="7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca"
Nov 03 16:39:49.234595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.234578    5605 scope.go:122] "RemoveContainer" containerID="7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca"
Nov 03 16:39:49.234898 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:49.234880    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca\": container with ID starting with 7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca not found: ID does not exist" containerID="7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca"
Nov 03 16:39:49.234961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.234908    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca"} err="failed to get container status \"7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca\": rpc error: code = NotFound desc = could not find container \"7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca\": container with ID starting with 7b88aab9b681abfdb267a1e083ec14532f2688c12c3d5b891ec2c5741b4956ca not found: ID does not exist"
Nov 03 16:39:49.234961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.234930    5605 scope.go:122] "RemoveContainer" containerID="8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc"
Nov 03 16:39:49.244439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.244415    5605 scope.go:122] "RemoveContainer" containerID="8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc"
Nov 03 16:39:49.244741 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:49.244700    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc\": container with ID starting with 8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc not found: ID does not exist" containerID="8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc"
Nov 03 16:39:49.244836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.244766    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc"} err="failed to get container status \"8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc\": rpc error: code = NotFound desc = could not find container \"8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc\": container with ID starting with 8be76d4433014759cc6799e5081342c6082645dbd0a6f8d41191bc04c98c80bc not found: ID does not exist"
Nov 03 16:39:49.244836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.244791    5605 scope.go:122] "RemoveContainer" containerID="03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7"
Nov 03 16:39:49.254691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.254672    5605 scope.go:122] "RemoveContainer" containerID="03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7"
Nov 03 16:39:49.255021 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:49.255000    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7\": container with ID starting with 03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7 not found: ID does not exist" containerID="03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7"
Nov 03 16:39:49.255114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.255027    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7"} err="failed to get container status \"03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7\": rpc error: code = NotFound desc = could not find container \"03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7\": container with ID starting with 03c39fe071e8707d501cfd56f2fda74868ba6c9d75bfda6c99b7de263f6bbca7 not found: ID does not exist"
Nov 03 16:39:49.400266 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.382530    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"]
Nov 03 16:39:49.400266 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.382802    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy" containerID="cri-o://d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474" gracePeriod=30
Nov 03 16:39:49.849560 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.849511    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"]
Nov 03 16:39:49.856247 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.856217    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-7nptr"]
Nov 03 16:39:49.974092 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.974057    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"]
Nov 03 16:39:49.977763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:49.977740    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-9kk5g"]
Nov 03 16:39:50.217915 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:50.217832    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"]
Nov 03 16:39:50.229941 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:50.229911    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-hkljp"]
Nov 03 16:39:50.310583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:50.310554    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" path="/var/lib/kubelet/pods/5c153a4f-9b14-4934-84a8-ba29826e20dd/volumes"
Nov 03 16:39:50.311096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:50.311071    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" path="/var/lib/kubelet/pods/5e21fcd1-f5a4-46cf-a8da-c340063f7727/volumes"
Nov 03 16:39:50.311539 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:50.311525    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" path="/var/lib/kubelet/pods/ab094755-d4d2-40bc-a411-e73e6fd3372f/volumes"
Nov 03 16:39:50.344862 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:50.344821    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"]
Nov 03 16:39:50.349183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:50.349153    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-547a38-openshift-default-cddffb669-fp7qn"]
Nov 03 16:39:51.438237 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438202    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"]
Nov 03 16:39:51.438836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438822    5605 cpu_manager.go:414] "containerMap: removing container" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy"
Nov 03 16:39:51.438891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438837    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy"
Nov 03 16:39:51.438891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438855    5605 cpu_manager.go:414] "containerMap: removing container" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy"
Nov 03 16:39:51.438891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438860    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy"
Nov 03 16:39:51.438891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438866    5605 cpu_manager.go:414] "containerMap: removing container" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy"
Nov 03 16:39:51.438891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438871    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy"
Nov 03 16:39:51.438891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438878    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy"
Nov 03 16:39:51.438891 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438882    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy"
Nov 03 16:39:51.439102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438982    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" containerName="istio-proxy"
Nov 03 16:39:51.439102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438992    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e21fcd1-f5a4-46cf-a8da-c340063f7727" containerName="istio-proxy"
Nov 03 16:39:51.439102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.438996    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c153a4f-9b14-4934-84a8-ba29826e20dd" containerName="istio-proxy"
Nov 03 16:39:51.439102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.439004    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab094755-d4d2-40bc-a411-e73e6fd3372f" containerName="istio-proxy"
Nov 03 16:39:51.444189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.444174    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.508757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.506804    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"]
Nov 03 16:39:51.515595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.515472    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qwd7\" (UniqueName: \"kubernetes.io/projected/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-kube-api-access-7qwd7\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.515807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.515658    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-model-cache\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.515881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.515849    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-dshm\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.515934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.515882    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tmp-dir\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.516001 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.515945    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-home\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.516001 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.515995    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tls-certs\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.548990 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.548962    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9635db0d\"/\"openshift-service-ca.crt\""
Nov 03 16:39:51.549122 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.549050    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9635db0d\"/\"kube-root-ca.crt\""
Nov 03 16:39:51.550171 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.550144    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9635db0d\"/\"e2e-lifecycle-model-9635db0d-kserve-self-signed-certs\""
Nov 03 16:39:51.550171 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.550165    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9635db0d\"/\"default-dockercfg-795dw\""
Nov 03 16:39:51.664753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.659632    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-model-cache\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.664753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.659941    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-dshm\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.664753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.659980    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tmp-dir\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.664753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.660045    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-home\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.664753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.660104    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tls-certs\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.664753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.660191    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qwd7\" (UniqueName: \"kubernetes.io/projected/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-kube-api-access-7qwd7\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.664753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.661423    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-model-cache\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.664753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.661990    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-home\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.664753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.662225    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tmp-dir\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.708811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.706590    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tls-certs\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.729757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.728018    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-dshm\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.751353 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.749515    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qwd7\" (UniqueName: \"kubernetes.io/projected/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-kube-api-access-7qwd7\") pod \"e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") " pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:51.758243 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:51.756935    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:52.152762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.152366    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"]
Nov 03 16:39:52.332752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.277714    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:39:52.332752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.279024    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:39:52.332752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.288836    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"]
Nov 03 16:39:52.495775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.486627    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-cc4dcd\""
Nov 03 16:39:52.495775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.487705    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-cc4dcd-dockercfg-4htrg\""
Nov 03 16:39:52.567748 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.563215    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2pk5\" (UniqueName: \"kubernetes.io/projected/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-kube-api-access-t2pk5\") pod \"payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b\" (UID: \"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:39:52.567748 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.563346    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b\" (UID: \"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:39:52.654309 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:52.647131    5605 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cd488c1_cb1d_42e0_bec6_be52b44f1c1f.slice/crio-b13afb1ab55204345920cb081892841d3f0faaaf451b9959528f71a0167fce21\": RecentStats: unable to find data in memory cache]"
Nov 03 16:39:52.657447 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.657422    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6dce1a44-2d2b-488a-ac77-ff556bf12027" path="/var/lib/kubelet/pods/6dce1a44-2d2b-488a-ac77-ff556bf12027/volumes"
Nov 03 16:39:52.657867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.657849    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"]
Nov 03 16:39:52.663571 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.663548    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"]
Nov 03 16:39:52.663654 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.663582    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"]
Nov 03 16:39:52.663708 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.663687    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:39:52.664129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.664107    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2pk5\" (UniqueName: \"kubernetes.io/projected/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-kube-api-access-t2pk5\") pod \"payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b\" (UID: \"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:39:52.664205 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.664190    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b\" (UID: \"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:39:52.665144 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.665121    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b\" (UID: \"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:39:52.724188 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.724161    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2pk5\" (UniqueName: \"kubernetes.io/projected/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-kube-api-access-t2pk5\") pod \"payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b\" (UID: \"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:39:52.728754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.728702    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:39:52.766167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.766133    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x2fr\" (UniqueName: \"kubernetes.io/projected/f315d524-c3e5-405a-85e6-e7ca8d2718cf-kube-api-access-6x2fr\") pod \"payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n\" (UID: \"f315d524-c3e5-405a-85e6-e7ca8d2718cf\") " pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:39:52.766341 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.766321    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f315d524-c3e5-405a-85e6-e7ca8d2718cf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n\" (UID: \"f315d524-c3e5-405a-85e6-e7ca8d2718cf\") " pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:39:52.867717 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.867677    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x2fr\" (UniqueName: \"kubernetes.io/projected/f315d524-c3e5-405a-85e6-e7ca8d2718cf-kube-api-access-6x2fr\") pod \"payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n\" (UID: \"f315d524-c3e5-405a-85e6-e7ca8d2718cf\") " pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:39:52.868330 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.868296    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f315d524-c3e5-405a-85e6-e7ca8d2718cf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n\" (UID: \"f315d524-c3e5-405a-85e6-e7ca8d2718cf\") " pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:39:52.870464 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.869209    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f315d524-c3e5-405a-85e6-e7ca8d2718cf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n\" (UID: \"f315d524-c3e5-405a-85e6-e7ca8d2718cf\") " pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:39:52.930518 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:52.930446    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x2fr\" (UniqueName: \"kubernetes.io/projected/f315d524-c3e5-405a-85e6-e7ca8d2718cf-kube-api-access-6x2fr\") pod \"payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n\" (UID: \"f315d524-c3e5-405a-85e6-e7ca8d2718cf\") " pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:39:53.029235 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.029202    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:39:53.151567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.151537    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"]
Nov 03 16:39:53.152691 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:39:53.152664    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc77fe985_3f03_4e0f_b4fe_325d2d3fb79d.slice/crio-0b06dec5fb3b7f3cc778f65bffc3756f0d5eac6b812aa725e37e6c598ea434b4 WatchSource:0}: Error finding container 0b06dec5fb3b7f3cc778f65bffc3756f0d5eac6b812aa725e37e6c598ea434b4: Status 404 returned error can't find the container with id 0b06dec5fb3b7f3cc778f65bffc3756f0d5eac6b812aa725e37e6c598ea434b4
Nov 03 16:39:53.318716 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.318676    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv" event={"ID":"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f","Type":"ContainerStarted","Data":"1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac"}
Nov 03 16:39:53.318905 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.318744    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv" event={"ID":"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f","Type":"ContainerStarted","Data":"b13afb1ab55204345920cb081892841d3f0faaaf451b9959528f71a0167fce21"}
Nov 03 16:39:53.322551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.322519    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b" event={"ID":"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d","Type":"ContainerStarted","Data":"d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac"}
Nov 03 16:39:53.322720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.322581    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:39:53.322720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.322614    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b" event={"ID":"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d","Type":"ContainerStarted","Data":"0b06dec5fb3b7f3cc778f65bffc3756f0d5eac6b812aa725e37e6c598ea434b4"}
Nov 03 16:39:53.440958 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.440932    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"]
Nov 03 16:39:53.441885 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.441847    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv" podStartSLOduration=2.441835049 podStartE2EDuration="2.441835049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:39:53.440212161 +0000 UTC m=+1679.264663429" watchObservedRunningTime="2026-11-03 16:39:53.441835049 +0000 UTC m=+1679.266286321"
Nov 03 16:39:53.443381 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:39:53.443355    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf315d524_c3e5_405a_85e6_e7ca8d2718cf.slice/crio-115a892f06fd65b755cf18d400f914df4a75bd9dd46eb461e2acf6b83127e70f WatchSource:0}: Error finding container 115a892f06fd65b755cf18d400f914df4a75bd9dd46eb461e2acf6b83127e70f: Status 404 returned error can't find the container with id 115a892f06fd65b755cf18d400f914df4a75bd9dd46eb461e2acf6b83127e70f
Nov 03 16:39:53.613751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.609661    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b" podStartSLOduration=1.609637486 podStartE2EDuration="1.609637486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:39:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:39:53.606299987 +0000 UTC m=+1679.430751259" watchObservedRunningTime="2026-11-03 16:39:53.609637486 +0000 UTC m=+1679.434088755"
Nov 03 16:39:53.763104 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.763055    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:39:53.844758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.842341    5605 patch_prober.go:29] interesting pod/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:39:53.844758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:53.842421    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:39:54.249309 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.244450    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:39:54.333690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.333661    5605 generic.go:356] "Generic (PLEG): container finished" podID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerID="bd57e88bbcb3fa578ac45961573ae740901070347ca9861f4730f0fcfe6ac73c" exitCode=0
Nov 03 16:39:54.333846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.333715    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" event={"ID":"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019","Type":"ContainerDied","Data":"bd57e88bbcb3fa578ac45961573ae740901070347ca9861f4730f0fcfe6ac73c"}
Nov 03 16:39:54.335204 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335181    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-data\") pod \"43263885-de5b-4337-848e-39e6bd270649\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") "
Nov 03 16:39:54.335284 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335224    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-kube-api-access-wcvg5\" (UniqueName: \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-kube-api-access-wcvg5\") pod \"43263885-de5b-4337-848e-39e6bd270649\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") "
Nov 03 16:39:54.335371 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335282    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-credential-socket\") pod \"43263885-de5b-4337-848e-39e6bd270649\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") "
Nov 03 16:39:54.335422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335378    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43263885-de5b-4337-848e-39e6bd270649-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43263885-de5b-4337-848e-39e6bd270649-istiod-ca-cert\") pod \"43263885-de5b-4337-848e-39e6bd270649\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") "
Nov 03 16:39:54.335796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335469    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-certs\") pod \"43263885-de5b-4337-848e-39e6bd270649\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") "
Nov 03 16:39:54.335796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335532    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-envoy\") pod \"43263885-de5b-4337-848e-39e6bd270649\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") "
Nov 03 16:39:54.335796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335650    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-istio-token\" (UniqueName: \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-istio-token\") pod \"43263885-de5b-4337-848e-39e6bd270649\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") "
Nov 03 16:39:54.335796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335719    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-socket\") pod \"43263885-de5b-4337-848e-39e6bd270649\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") "
Nov 03 16:39:54.335796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335784    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-credential-socket" pod "43263885-de5b-4337-848e-39e6bd270649" (UID: "43263885-de5b-4337-848e-39e6bd270649"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.336069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335799    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/43263885-de5b-4337-848e-39e6bd270649-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43263885-de5b-4337-848e-39e6bd270649-istio-podinfo\") pod \"43263885-de5b-4337-848e-39e6bd270649\" (UID: \"43263885-de5b-4337-848e-39e6bd270649\") "
Nov 03 16:39:54.336069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.335801    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-data" pod "43263885-de5b-4337-848e-39e6bd270649" (UID: "43263885-de5b-4337-848e-39e6bd270649"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.336169 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.336110    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43263885-de5b-4337-848e-39e6bd270649-istiod-ca-cert" pod "43263885-de5b-4337-848e-39e6bd270649" (UID: "43263885-de5b-4337-848e-39e6bd270649"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:54.336447 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.336425    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43263885-de5b-4337-848e-39e6bd270649-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.336614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.336592    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.336716 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.336620    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.336716 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.336446    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-certs" pod "43263885-de5b-4337-848e-39e6bd270649" (UID: "43263885-de5b-4337-848e-39e6bd270649"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.336716 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.336606    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-socket" pod "43263885-de5b-4337-848e-39e6bd270649" (UID: "43263885-de5b-4337-848e-39e6bd270649"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.339099 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.339070    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n" event={"ID":"f315d524-c3e5-405a-85e6-e7ca8d2718cf","Type":"ContainerStarted","Data":"0d73d8a9bf92e35ab3d672dbc1ce673417704405922d549d0c0382e52a9c1909"}
Nov 03 16:39:54.339213 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.339125    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n" event={"ID":"f315d524-c3e5-405a-85e6-e7ca8d2718cf","Type":"ContainerStarted","Data":"115a892f06fd65b755cf18d400f914df4a75bd9dd46eb461e2acf6b83127e70f"}
Nov 03 16:39:54.339213 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.339173    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:39:54.339973 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.339950    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43263885-de5b-4337-848e-39e6bd270649-istio-podinfo" pod "43263885-de5b-4337-848e-39e6bd270649" (UID: "43263885-de5b-4337-848e-39e6bd270649"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:54.340560 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.340494    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-istio-token" pod "43263885-de5b-4337-848e-39e6bd270649" (UID: "43263885-de5b-4337-848e-39e6bd270649"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.340560 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.340531    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-kube-api-access-wcvg5" pod "43263885-de5b-4337-848e-39e6bd270649" (UID: "43263885-de5b-4337-848e-39e6bd270649"). InnerVolumeSpecName "kube-api-access-wcvg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.341114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.341094    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-envoy" pod "43263885-de5b-4337-848e-39e6bd270649" (UID: "43263885-de5b-4337-848e-39e6bd270649"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.342946 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.342924    5605 generic.go:356] "Generic (PLEG): container finished" podID="43263885-de5b-4337-848e-39e6bd270649" containerID="1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9" exitCode=0
Nov 03 16:39:54.343041 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.343026    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" event={"ID":"43263885-de5b-4337-848e-39e6bd270649","Type":"ContainerDied","Data":"1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9"}
Nov 03 16:39:54.343102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.343076    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp" event={"ID":"43263885-de5b-4337-848e-39e6bd270649","Type":"ContainerDied","Data":"8b967cb1441728ce9709c9d5cd995ef151c71cc2bf96c989603b5bcde6e1cef0"}
Nov 03 16:39:54.343146 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.343105    5605 scope.go:122] "RemoveContainer" containerID="1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9"
Nov 03 16:39:54.343238 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.343226    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"
Nov 03 16:39:54.346246 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.346225    5605 generic.go:356] "Generic (PLEG): container finished" podID="922c42d0-8663-4e36-99ad-4f8326b16823" containerID="5e6c091adb23699d176761af10ad19a254dfa027bb75aef25e85e6460e73cf46" exitCode=0
Nov 03 16:39:54.346331 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.346319    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" event={"ID":"922c42d0-8663-4e36-99ad-4f8326b16823","Type":"ContainerDied","Data":"5e6c091adb23699d176761af10ad19a254dfa027bb75aef25e85e6460e73cf46"}
Nov 03 16:39:54.348717 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.348635    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:39:54.351609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.351590    5605 generic.go:356] "Generic (PLEG): container finished" podID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerID="85807ecc93933e92dbdffdbac1b60541f94b80bd121599bdfabb11c715dd4615" exitCode=0
Nov 03 16:39:54.351708 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.351696    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" event={"ID":"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066","Type":"ContainerDied","Data":"85807ecc93933e92dbdffdbac1b60541f94b80bd121599bdfabb11c715dd4615"}
Nov 03 16:39:54.358658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.358636    5605 scope.go:122] "RemoveContainer" containerID="1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9"
Nov 03 16:39:54.364776 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:54.360297    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9\": container with ID starting with 1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9 not found: ID does not exist" containerID="1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9"
Nov 03 16:39:54.364776 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.360324    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9"} err="failed to get container status \"1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9\": rpc error: code = NotFound desc = could not find container \"1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9\": container with ID starting with 1b7ab238139662861b27ab3504461d96da76a5ed5f5cefeb7b2835e3c6627bf9 not found: ID does not exist"
Nov 03 16:39:54.364776 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.360341    5605 scope.go:122] "RemoveContainer" containerID="85807ecc93933e92dbdffdbac1b60541f94b80bd121599bdfabb11c715dd4615"
Nov 03 16:39:54.364776 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.361422    5605 generic.go:356] "Generic (PLEG): container finished" podID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerID="7554893895a7c1e24e2bba1aff977813e01e0bddb92bc4da89e6440a7f1fadd8" exitCode=0
Nov 03 16:39:54.364776 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.362350    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" event={"ID":"c89ea04b-5366-4ba9-b3a3-30926721c6fb","Type":"ContainerDied","Data":"7554893895a7c1e24e2bba1aff977813e01e0bddb92bc4da89e6440a7f1fadd8"}
Nov 03 16:39:54.376928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.376908    5605 scope.go:122] "RemoveContainer" containerID="975a84732baa998eb1d16061be20b77a36f077f30bf0ef318db3e284981057af"
Nov 03 16:39:54.437446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.437420    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istiod-ca-cert\") pod \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") "
Nov 03 16:39:54.437619 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.437602    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-credential-socket\") pod \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") "
Nov 03 16:39:54.437683 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.437664    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-token\" (UniqueName: \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-token\") pod \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") "
Nov 03 16:39:54.437757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.437741    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-envoy\") pod \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") "
Nov 03 16:39:54.437811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.437788    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-certs\") pod \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") "
Nov 03 16:39:54.437875 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.437833    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istiod-ca-cert" pod "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" (UID: "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:54.437875 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.437869    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-socket\") pod \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") "
Nov 03 16:39:54.437972 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.437912    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-data\") pod \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") "
Nov 03 16:39:54.437972 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.437948    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-kube-api-access-487zc\" (UniqueName: \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-kube-api-access-487zc\") pod \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") "
Nov 03 16:39:54.438074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.438024    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-podinfo\") pod \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\" (UID: \"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066\") "
Nov 03 16:39:54.438138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.438113    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-certs" pod "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" (UID: "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.438255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.438239    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-data" pod "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" (UID: "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.438389 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.438370    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-socket" pod "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" (UID: "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.438830 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.438807    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-credential-socket" pod "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" (UID: "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.439309 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.439291    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43263885-de5b-4337-848e-39e6bd270649-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.439676 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.439656    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcvg5\" (UniqueName: \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-kube-api-access-wcvg5\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.439843 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.439825    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.439967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.439952    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.440083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.440069    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.440198 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.440182    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.440303 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.440290    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.440386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.440374    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43263885-de5b-4337-848e-39e6bd270649-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.440468 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.440456    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.440555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.440543    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43263885-de5b-4337-848e-39e6bd270649-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.443511 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.443406    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-envoy" pod "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" (UID: "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.444113 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.443671    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-podinfo" pod "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" (UID: "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:54.444113 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.443224    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-kube-api-access-487zc" pod "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" (UID: "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066"). InnerVolumeSpecName "kube-api-access-487zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.444113 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.444075    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-token" pod "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" (UID: "ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.453278 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.453261    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:39:54.541464 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.541441    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-kube-api-access-bl8r5\" (UniqueName: \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-kube-api-access-bl8r5\") pod \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") "
Nov 03 16:39:54.541575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.541490    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-certs\") pod \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") "
Nov 03 16:39:54.541656 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.541641    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-envoy\") pod \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") "
Nov 03 16:39:54.541746 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.541706    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istiod-ca-cert\") pod \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") "
Nov 03 16:39:54.541820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.541794    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-credential-socket\") pod \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") "
Nov 03 16:39:54.541875 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.541832    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-data\") pod \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") "
Nov 03 16:39:54.541924 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.541873    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-podinfo\") pod \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") "
Nov 03 16:39:54.541974 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.541943    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-token\" (UniqueName: \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-token\") pod \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") "
Nov 03 16:39:54.542371 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.542331    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-certs" pod "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" (UID: "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.542587 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.542563    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-credential-socket" pod "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" (UID: "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.542844 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.542824    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istiod-ca-cert" pod "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" (UID: "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:54.542986 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.542836    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-socket\") pod \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\" (UID: \"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019\") "
Nov 03 16:39:54.543218 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.543083    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-socket" pod "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" (UID: "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.543294 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.543227    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-data" pod "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" (UID: "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.544508 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.544484    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-487zc\" (UniqueName: \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-kube-api-access-487zc\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.544634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.544619    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.544761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.544747    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.544865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.544853    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.544957 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.544942    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.545051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.545038    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.545141 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.545131    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.545344 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.545315    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-kube-api-access-bl8r5" pod "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" (UID: "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019"). InnerVolumeSpecName "kube-api-access-bl8r5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.545481 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.545466    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.545609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.545596    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.545785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.545488    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-token" pod "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" (UID: "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.545870 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.545847    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.545954 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.545932    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-podinfo" pod "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" (UID: "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:54.546048 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.546031    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-envoy" pod "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" (UID: "aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.573202 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.573167    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"]
Nov 03 16:39:54.578110 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.578082    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-jjtgp"]
Nov 03 16:39:54.622988 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.622949    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:39:54.628691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.628668    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:39:54.646509 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.646484    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-data\") pod \"922c42d0-8663-4e36-99ad-4f8326b16823\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") "
Nov 03 16:39:54.646646 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.646527    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-istio-token\" (UniqueName: \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-istio-token\") pod \"922c42d0-8663-4e36-99ad-4f8326b16823\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") "
Nov 03 16:39:54.646646 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.646594    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-certs\") pod \"922c42d0-8663-4e36-99ad-4f8326b16823\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") "
Nov 03 16:39:54.646646 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.646619    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/922c42d0-8663-4e36-99ad-4f8326b16823-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922c42d0-8663-4e36-99ad-4f8326b16823-istiod-ca-cert\") pod \"922c42d0-8663-4e36-99ad-4f8326b16823\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") "
Nov 03 16:39:54.646770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.646759    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-kube-api-access-grsr6\" (UniqueName: \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-kube-api-access-grsr6\") pod \"922c42d0-8663-4e36-99ad-4f8326b16823\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") "
Nov 03 16:39:54.646821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.646787    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-credential-socket\") pod \"922c42d0-8663-4e36-99ad-4f8326b16823\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") "
Nov 03 16:39:54.646859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.646823    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-socket\") pod \"922c42d0-8663-4e36-99ad-4f8326b16823\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") "
Nov 03 16:39:54.646915 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.646893    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-envoy\") pod \"922c42d0-8663-4e36-99ad-4f8326b16823\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") "
Nov 03 16:39:54.646960 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.646949    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/922c42d0-8663-4e36-99ad-4f8326b16823-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922c42d0-8663-4e36-99ad-4f8326b16823-istio-podinfo\") pod \"922c42d0-8663-4e36-99ad-4f8326b16823\" (UID: \"922c42d0-8663-4e36-99ad-4f8326b16823\") "
Nov 03 16:39:54.647369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.647354    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl8r5\" (UniqueName: \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-kube-api-access-bl8r5\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.647405 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.647372    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.647405 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.647381    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.647405 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.647392    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.647999 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.647974    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-socket" pod "922c42d0-8663-4e36-99ad-4f8326b16823" (UID: "922c42d0-8663-4e36-99ad-4f8326b16823"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.648455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.648390    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-certs" pod "922c42d0-8663-4e36-99ad-4f8326b16823" (UID: "922c42d0-8663-4e36-99ad-4f8326b16823"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.648455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.648406    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-credential-socket" pod "922c42d0-8663-4e36-99ad-4f8326b16823" (UID: "922c42d0-8663-4e36-99ad-4f8326b16823"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.648455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.648440    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/922c42d0-8663-4e36-99ad-4f8326b16823-istiod-ca-cert" pod "922c42d0-8663-4e36-99ad-4f8326b16823" (UID: "922c42d0-8663-4e36-99ad-4f8326b16823"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:54.648864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.648584    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-data" pod "922c42d0-8663-4e36-99ad-4f8326b16823" (UID: "922c42d0-8663-4e36-99ad-4f8326b16823"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.650347 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.650326    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/922c42d0-8663-4e36-99ad-4f8326b16823-istio-podinfo" pod "922c42d0-8663-4e36-99ad-4f8326b16823" (UID: "922c42d0-8663-4e36-99ad-4f8326b16823"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:54.650425 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.650372    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-kube-api-access-grsr6" pod "922c42d0-8663-4e36-99ad-4f8326b16823" (UID: "922c42d0-8663-4e36-99ad-4f8326b16823"). InnerVolumeSpecName "kube-api-access-grsr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.651253 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.651232    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-istio-token" pod "922c42d0-8663-4e36-99ad-4f8326b16823" (UID: "922c42d0-8663-4e36-99ad-4f8326b16823"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.651308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.651279    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-envoy" pod "922c42d0-8663-4e36-99ad-4f8326b16823" (UID: "922c42d0-8663-4e36-99ad-4f8326b16823"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.748110 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748083    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-certs\") pod \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") "
Nov 03 16:39:54.748234 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748141    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-socket\") pod \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") "
Nov 03 16:39:54.748234 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748172    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istiod-ca-cert\") pod \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") "
Nov 03 16:39:54.748234 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748197    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-podinfo\") pod \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") "
Nov 03 16:39:54.748234 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748220    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-envoy\") pod \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") "
Nov 03 16:39:54.748381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748296    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-data\") pod \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") "
Nov 03 16:39:54.748381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748336    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-kube-api-access-nv6wm\" (UniqueName: \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-kube-api-access-nv6wm\") pod \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") "
Nov 03 16:39:54.748445 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748380    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-certs" pod "c89ea04b-5366-4ba9-b3a3-30926721c6fb" (UID: "c89ea04b-5366-4ba9-b3a3-30926721c6fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.748993 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748506    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-token\" (UniqueName: \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-token\") pod \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") "
Nov 03 16:39:54.748993 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748542    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-credential-socket\") pod \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\" (UID: \"c89ea04b-5366-4ba9-b3a3-30926721c6fb\") "
Nov 03 16:39:54.748993 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748544    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istiod-ca-cert" pod "c89ea04b-5366-4ba9-b3a3-30926721c6fb" (UID: "c89ea04b-5366-4ba9-b3a3-30926721c6fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:54.748993 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748799    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-socket" pod "c89ea04b-5366-4ba9-b3a3-30926721c6fb" (UID: "c89ea04b-5366-4ba9-b3a3-30926721c6fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.748993 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.748971    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-data" pod "c89ea04b-5366-4ba9-b3a3-30926721c6fb" (UID: "c89ea04b-5366-4ba9-b3a3-30926721c6fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.749236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749052    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-credential-socket" pod "c89ea04b-5366-4ba9-b3a3-30926721c6fb" (UID: "c89ea04b-5366-4ba9-b3a3-30926721c6fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.749497 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749473    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grsr6\" (UniqueName: \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-kube-api-access-grsr6\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749534 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749507    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749534 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749523    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749592 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749538    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749592 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749553    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749592 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749567    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749592 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749581    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749594    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749610    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749626    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922c42d0-8663-4e36-99ad-4f8326b16823-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749640    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749654    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922c42d0-8663-4e36-99ad-4f8326b16823-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749667    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922c42d0-8663-4e36-99ad-4f8326b16823-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.749703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.749683    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922c42d0-8663-4e36-99ad-4f8326b16823-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.750030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.750012    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:39:54.753024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.752010    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-podinfo" pod "c89ea04b-5366-4ba9-b3a3-30926721c6fb" (UID: "c89ea04b-5366-4ba9-b3a3-30926721c6fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:54.753024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.752024    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-envoy" pod "c89ea04b-5366-4ba9-b3a3-30926721c6fb" (UID: "c89ea04b-5366-4ba9-b3a3-30926721c6fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.753024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.752074    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-kube-api-access-nv6wm" pod "c89ea04b-5366-4ba9-b3a3-30926721c6fb" (UID: "c89ea04b-5366-4ba9-b3a3-30926721c6fb"). InnerVolumeSpecName "kube-api-access-nv6wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.753312 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.753290    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-token" pod "c89ea04b-5366-4ba9-b3a3-30926721c6fb" (UID: "c89ea04b-5366-4ba9-b3a3-30926721c6fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.855833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.855213    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-credential-socket\") pod \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") "
Nov 03 16:39:54.855833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.855760    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-socket\") pod \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") "
Nov 03 16:39:54.856189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.855862    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istiod-ca-cert\") pod \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") "
Nov 03 16:39:54.856189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.855889    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-kube-api-access-mxqvn\" (UniqueName: \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-kube-api-access-mxqvn\") pod \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") "
Nov 03 16:39:54.856189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.856032    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-podinfo\") pod \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") "
Nov 03 16:39:54.856189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.856067    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-envoy\") pod \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") "
Nov 03 16:39:54.856189 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.856104    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-token\" (UniqueName: \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-token\") pod \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") "
Nov 03 16:39:54.856667 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.856249    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-certs\") pod \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") "
Nov 03 16:39:54.856667 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.856281    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-data\") pod \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\" (UID: \"4eb423ba-e1b2-4f9d-bae0-8b189990ec26\") "
Nov 03 16:39:54.859074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.855614    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-credential-socket" pod "4eb423ba-e1b2-4f9d-bae0-8b189990ec26" (UID: "4eb423ba-e1b2-4f9d-bae0-8b189990ec26"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.859074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.858035    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-socket" pod "4eb423ba-e1b2-4f9d-bae0-8b189990ec26" (UID: "4eb423ba-e1b2-4f9d-bae0-8b189990ec26"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.859074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.858747    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istiod-ca-cert" pod "4eb423ba-e1b2-4f9d-bae0-8b189990ec26" (UID: "4eb423ba-e1b2-4f9d-bae0-8b189990ec26"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:39:54.859074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.858994    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-certs" pod "4eb423ba-e1b2-4f9d-bae0-8b189990ec26" (UID: "4eb423ba-e1b2-4f9d-bae0-8b189990ec26"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.859074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.859022    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-data" pod "4eb423ba-e1b2-4f9d-bae0-8b189990ec26" (UID: "4eb423ba-e1b2-4f9d-bae0-8b189990ec26"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.859384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.859255    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.859384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.859276    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.859518 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.859444    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.859518 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.859466    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.859518 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.859481    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.859518 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.859495    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.859518 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.859510    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.859828 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.859524    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c89ea04b-5366-4ba9-b3a3-30926721c6fb-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.859828 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.859539    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv6wm\" (UniqueName: \"kubernetes.io/projected/c89ea04b-5366-4ba9-b3a3-30926721c6fb-kube-api-access-nv6wm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.860605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.860579    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-token" pod "4eb423ba-e1b2-4f9d-bae0-8b189990ec26" (UID: "4eb423ba-e1b2-4f9d-bae0-8b189990ec26"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.861999 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.861961    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-podinfo" pod "4eb423ba-e1b2-4f9d-bae0-8b189990ec26" (UID: "4eb423ba-e1b2-4f9d-bae0-8b189990ec26"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:39:54.863047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.863016    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-envoy" pod "4eb423ba-e1b2-4f9d-bae0-8b189990ec26" (UID: "4eb423ba-e1b2-4f9d-bae0-8b189990ec26"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:39:54.863185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.863102    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-kube-api-access-mxqvn" pod "4eb423ba-e1b2-4f9d-bae0-8b189990ec26" (UID: "4eb423ba-e1b2-4f9d-bae0-8b189990ec26"). InnerVolumeSpecName "kube-api-access-mxqvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:39:54.873627 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.873589    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n" podStartSLOduration=2.873576457 podStartE2EDuration="2.873576457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:39:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:39:54.87095439 +0000 UTC m=+1680.695405663" watchObservedRunningTime="2026-11-03 16:39:54.873576457 +0000 UTC m=+1680.698027724"
Nov 03 16:39:54.961172 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.961136    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.961353 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.961198    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.961353 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.961215    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:54.961353 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:54.961230    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxqvn\" (UniqueName: \"kubernetes.io/projected/4eb423ba-e1b2-4f9d-bae0-8b189990ec26-kube-api-access-mxqvn\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:39:55.016639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.016564    5605 scope.go:122] "RemoveContainer" containerID="7554893895a7c1e24e2bba1aff977813e01e0bddb92bc4da89e6440a7f1fadd8"
Nov 03 16:39:55.029445 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.029415    5605 scope.go:122] "RemoveContainer" containerID="5e6c091adb23699d176761af10ad19a254dfa027bb75aef25e85e6460e73cf46"
Nov 03 16:39:55.370229 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.370185    5605 generic.go:356] "Generic (PLEG): container finished" podID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerID="d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474" exitCode=0
Nov 03 16:39:55.370387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.370237    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"
Nov 03 16:39:55.370387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.370325    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" event={"ID":"4eb423ba-e1b2-4f9d-bae0-8b189990ec26","Type":"ContainerDied","Data":"d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474"}
Nov 03 16:39:55.370387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.370385    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f" event={"ID":"4eb423ba-e1b2-4f9d-bae0-8b189990ec26","Type":"ContainerDied","Data":"6881a61e1feb6bffc74fd48f7d3fafdec219bedbf15f1af5d4b53cc25d2d7bc8"}
Nov 03 16:39:55.370566 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.370407    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t" event={"ID":"922c42d0-8663-4e36-99ad-4f8326b16823","Type":"ContainerDied","Data":"9e312ffd4ecd7136a3690e14e60089b96feb9b2aba5948c63ccda4b1dfc13625"}
Nov 03 16:39:55.370566 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.370436    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7" event={"ID":"ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066","Type":"ContainerDied","Data":"ea116b30c0a183a50b518252e289e6a69a71c00aa05e35ca27e854e81f9a4f3e"}
Nov 03 16:39:55.370566 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.370468    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns" event={"ID":"c89ea04b-5366-4ba9-b3a3-30926721c6fb","Type":"ContainerDied","Data":"8d379f2113efefda8d3755f2cc7b66f3ead29ead1230de17ba3bd9836be658a2"}
Nov 03 16:39:55.370566 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.370476    5605 scope.go:122] "RemoveContainer" containerID="d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474"
Nov 03 16:39:55.372623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.372591    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"
Nov 03 16:39:55.372791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.372691    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"
Nov 03 16:39:55.372864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.372809    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k" event={"ID":"aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019","Type":"ContainerDied","Data":"7bd0c1295679b81acb98a00758d4549afe163510b7c364ba5a74596b0933afb8"}
Nov 03 16:39:55.373763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.373742    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"
Nov 03 16:39:55.425750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.379682    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"
Nov 03 16:39:55.425750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.415599    5605 scope.go:122] "RemoveContainer" containerID="d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474"
Nov 03 16:39:55.425750 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:39:55.416301    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474\": container with ID starting with d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474 not found: ID does not exist" containerID="d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474"
Nov 03 16:39:55.425750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.416338    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474"} err="failed to get container status \"d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474\": rpc error: code = NotFound desc = could not find container \"d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474\": container with ID starting with d1aa96a72addc01569cb716274ccea5af52555e243c42900d6574fbd5d9bc474 not found: ID does not exist"
Nov 03 16:39:55.425750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.416363    5605 scope.go:122] "RemoveContainer" containerID="bd57e88bbcb3fa578ac45961573ae740901070347ca9861f4730f0fcfe6ac73c"
Nov 03 16:39:55.710187 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.710100    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"]
Nov 03 16:39:55.713502 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:55.713480    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-kx98t"]
Nov 03 16:39:56.108445 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.108399    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"]
Nov 03 16:39:56.111112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.111085    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-l6jj7"]
Nov 03 16:39:56.222750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.222690    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"]
Nov 03 16:39:56.225664 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.225639    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-xdn4f"]
Nov 03 16:39:56.318872 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.318841    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43263885-de5b-4337-848e-39e6bd270649" path="/var/lib/kubelet/pods/43263885-de5b-4337-848e-39e6bd270649/volumes"
Nov 03 16:39:56.319366 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.319343    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" path="/var/lib/kubelet/pods/4eb423ba-e1b2-4f9d-bae0-8b189990ec26/volumes"
Nov 03 16:39:56.319793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.319781    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" path="/var/lib/kubelet/pods/922c42d0-8663-4e36-99ad-4f8326b16823/volumes"
Nov 03 16:39:56.320626 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.320608    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" path="/var/lib/kubelet/pods/ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066/volumes"
Nov 03 16:39:56.342125 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.342093    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"]
Nov 03 16:39:56.350849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.350823    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-ltqns"]
Nov 03 16:39:56.469682 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.469605    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"]
Nov 03 16:39:56.475918 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:56.475893    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-2bb620-openshift-default-5f5cd474fc-krs9k"]
Nov 03 16:39:58.310547 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:58.310513    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" path="/var/lib/kubelet/pods/aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019/volumes"
Nov 03 16:39:58.311051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:39:58.311035    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" path="/var/lib/kubelet/pods/c89ea04b-5366-4ba9-b3a3-30926721c6fb/volumes"
Nov 03 16:40:01.443710 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:01.443677    5605 generic.go:356] "Generic (PLEG): container finished" podID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerID="e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679" exitCode=0
Nov 03 16:40:01.444150 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:01.443755    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" event={"ID":"41871919-9219-49c2-abd8-6a38e7e32d2e","Type":"ContainerDied","Data":"e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679"}
Nov 03 16:40:01.444192 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:01.444166    5605 scope.go:122] "RemoveContainer" containerID="e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679"
Nov 03 16:40:01.445852 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:01.445815    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:01.445988 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:01.445904    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:01.445988 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:01.445949    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:01.757208 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:01.757113    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:40:01.770113 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:01.770079    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:40:02.452061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:02.452024    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" event={"ID":"41871919-9219-49c2-abd8-6a38e7e32d2e","Type":"ContainerStarted","Data":"be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35"}
Nov 03 16:40:02.464188 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:02.464162    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:40:03.107981 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:03.107938    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:40:03.109583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:03.109555    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 03 16:40:03.109759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:03.109605    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 03 16:40:03.414667 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:40:03.414575    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:40:04.108311 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:04.108268    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 03 16:40:04.108713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:04.108337    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 03 16:40:04.365881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:04.365792    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:40:05.007965 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:05.007922    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:40:05.108857 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:05.108819    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 03 16:40:05.109240 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:05.108885    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 03 16:40:05.385064 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:05.385033    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:40:06.108264 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:06.108222    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 03 16:40:06.108458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:06.108284    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 03 16:40:07.108169 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:07.108139    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 03 16:40:07.108630 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:07.108606    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 03 16:40:08.113089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:08.113058    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:40:08.114166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:08.114147    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 16:40:14.842025 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.841989    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"]
Nov 03 16:40:14.842529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842516    5605 cpu_manager.go:414] "containerMap: removing container" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy"
Nov 03 16:40:14.842581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842531    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy"
Nov 03 16:40:14.842581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842541    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy"
Nov 03 16:40:14.842581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842547    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy"
Nov 03 16:40:14.842581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842564    5605 cpu_manager.go:414] "containerMap: removing container" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy"
Nov 03 16:40:14.842581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842570    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy"
Nov 03 16:40:14.842581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842581    5605 cpu_manager.go:414] "containerMap: removing container" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842587    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842595    5605 cpu_manager.go:414] "containerMap: removing container" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842601    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842632    5605 cpu_manager.go:414] "containerMap: removing container" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842636    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842715    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa9c2e2d-9e39-434c-b2b5-db3dd5cb0019" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842738    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="43263885-de5b-4337-848e-39e6bd270649" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842746    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842753    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="c89ea04b-5366-4ba9-b3a3-30926721c6fb" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842759    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="4eb423ba-e1b2-4f9d-bae0-8b189990ec26" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842767    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="922c42d0-8663-4e36-99ad-4f8326b16823" containerName="istio-proxy"
Nov 03 16:40:14.842805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.842780    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy"
Nov 03 16:40:14.846267 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.846248    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:14.920796 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.915122    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-cb494a-openshift-default-dockercfg-97r7g\""
Nov 03 16:40:14.995213 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.978663    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"]
Nov 03 16:40:14.998871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.998763    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt99k\" (UniqueName: \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-kube-api-access-dt99k\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:14.999065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.998910    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-envoy\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:14.999065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.999018    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-certs\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:14.999201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.999147    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-socket\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:14.999259 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.999197    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-credential-socket\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:14.999259 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.999236    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-podinfo\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:14.999403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.999263    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-token\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:14.999403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.999299    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afb0f028-ecde-48ed-92b0-39276f7306e7-istiod-ca-cert\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:14.999403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:14.999340    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-data\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.108134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.108040    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt99k\" (UniqueName: \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-kube-api-access-dt99k\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.108822 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.108794    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-envoy\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.109428 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.109397    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-certs\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.110127 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.110107    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-socket\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.110335 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.110319    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-credential-socket\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.110791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.110771    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-podinfo\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.110948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.110929    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-token\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.111136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.111119    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afb0f028-ecde-48ed-92b0-39276f7306e7-istiod-ca-cert\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.111316 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.111288    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-data\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.114506 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.110452    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-socket\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.114681 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.109982    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-certs\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.115576 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.112367    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-data\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.115795 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.110688    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-credential-socket\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.115922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.113356    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afb0f028-ecde-48ed-92b0-39276f7306e7-istiod-ca-cert\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.121639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.121613    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-podinfo\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.131904 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.131833    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-envoy\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.174786 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.174717    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt99k\" (UniqueName: \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-kube-api-access-dt99k\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.175256 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.175239    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-token\") pod \"e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") " pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.199491 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.199462    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:15.645063 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:40:15.645035    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb0f028_ecde_48ed_92b0_39276f7306e7.slice/crio-000ca327e07cccef2bd355040e490312ba84764bf8e416c0da425a108c6c7df8 WatchSource:0}: Error finding container 000ca327e07cccef2bd355040e490312ba84764bf8e416c0da425a108c6c7df8: Status 404 returned error can't find the container with id 000ca327e07cccef2bd355040e490312ba84764bf8e416c0da425a108c6c7df8
Nov 03 16:40:15.646116 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.646095    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"]
Nov 03 16:40:15.648157 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.648117    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:15.648258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.648186    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:15.648258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:15.648214    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:16.567049 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:16.567011    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" event={"ID":"afb0f028-ecde-48ed-92b0-39276f7306e7","Type":"ContainerStarted","Data":"94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981"}
Nov 03 16:40:16.567440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:16.567083    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" event={"ID":"afb0f028-ecde-48ed-92b0-39276f7306e7","Type":"ContainerStarted","Data":"000ca327e07cccef2bd355040e490312ba84764bf8e416c0da425a108c6c7df8"}
Nov 03 16:40:16.751507 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:16.751457    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" podStartSLOduration=2.751443338 podStartE2EDuration="2.751443338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:40:16.75080601 +0000 UTC m=+1702.575257282" watchObservedRunningTime="2026-11-03 16:40:16.751443338 +0000 UTC m=+1702.575894609"
Nov 03 16:40:17.200134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:17.200102    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:17.205611 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:17.205587    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:17.574305 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:17.574222    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:17.575429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:17.575408    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:40:23.789204 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.789170    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"]
Nov 03 16:40:23.789985 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.789963    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy"
Nov 03 16:40:23.790100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.789987    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea6ffa84-a0a2-4fc5-ac4a-ceff5a1d2066" containerName="istio-proxy"
Nov 03 16:40:23.793842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.793821    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:40:23.856439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.856410    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"]
Nov 03 16:40:23.868791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.868763    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea72c89-5ec4-4822-be54-be0b1288d62c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl\" (UID: \"8ea72c89-5ec4-4822-be54-be0b1288d62c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:40:23.868941 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.868803    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z7d6\" (UniqueName: \"kubernetes.io/projected/8ea72c89-5ec4-4822-be54-be0b1288d62c-kube-api-access-2z7d6\") pod \"payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl\" (UID: \"8ea72c89-5ec4-4822-be54-be0b1288d62c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:40:23.913018 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.912984    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-cb494a-dockercfg-5djdw\""
Nov 03 16:40:23.926603 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.913426    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-cb494a\""
Nov 03 16:40:23.949887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.949847    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"]
Nov 03 16:40:23.987750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.970430    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea72c89-5ec4-4822-be54-be0b1288d62c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl\" (UID: \"8ea72c89-5ec4-4822-be54-be0b1288d62c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:40:23.988036 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.988010    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z7d6\" (UniqueName: \"kubernetes.io/projected/8ea72c89-5ec4-4822-be54-be0b1288d62c-kube-api-access-2z7d6\") pod \"payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl\" (UID: \"8ea72c89-5ec4-4822-be54-be0b1288d62c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:40:24.006752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.986546    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea72c89-5ec4-4822-be54-be0b1288d62c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl\" (UID: \"8ea72c89-5ec4-4822-be54-be0b1288d62c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:40:24.007046 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:23.999049    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:40:24.026771 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.024575    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"]
Nov 03 16:40:24.064508 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.064427    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z7d6\" (UniqueName: \"kubernetes.io/projected/8ea72c89-5ec4-4822-be54-be0b1288d62c-kube-api-access-2z7d6\") pod \"payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl\" (UID: \"8ea72c89-5ec4-4822-be54-be0b1288d62c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:40:24.089944 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.089906    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2e4bfb6-f892-4dae-8e86-36af9c6def94-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4\" (UID: \"c2e4bfb6-f892-4dae-8e86-36af9c6def94\") " pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:40:24.090139 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.090029    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pwff\" (UniqueName: \"kubernetes.io/projected/c2e4bfb6-f892-4dae-8e86-36af9c6def94-kube-api-access-6pwff\") pod \"payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4\" (UID: \"c2e4bfb6-f892-4dae-8e86-36af9c6def94\") " pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:40:24.107551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.107518    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:40:24.260255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.238348    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pwff\" (UniqueName: \"kubernetes.io/projected/c2e4bfb6-f892-4dae-8e86-36af9c6def94-kube-api-access-6pwff\") pod \"payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4\" (UID: \"c2e4bfb6-f892-4dae-8e86-36af9c6def94\") " pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:40:24.260255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.238503    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2e4bfb6-f892-4dae-8e86-36af9c6def94-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4\" (UID: \"c2e4bfb6-f892-4dae-8e86-36af9c6def94\") " pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:40:24.260255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.239191    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2e4bfb6-f892-4dae-8e86-36af9c6def94-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4\" (UID: \"c2e4bfb6-f892-4dae-8e86-36af9c6def94\") " pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:40:24.302290 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.302225    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pwff\" (UniqueName: \"kubernetes.io/projected/c2e4bfb6-f892-4dae-8e86-36af9c6def94-kube-api-access-6pwff\") pod \"payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4\" (UID: \"c2e4bfb6-f892-4dae-8e86-36af9c6def94\") " pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:40:24.354558 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.354529    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:40:24.566007 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.565977    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"]
Nov 03 16:40:24.566447 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:40:24.566422    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea72c89_5ec4_4822_be54_be0b1288d62c.slice/crio-d377dd7cd62fc77485a113deb1de9f7e654e33443ee90fafb5810b164ebe3b6d WatchSource:0}: Error finding container d377dd7cd62fc77485a113deb1de9f7e654e33443ee90fafb5810b164ebe3b6d: Status 404 returned error can't find the container with id d377dd7cd62fc77485a113deb1de9f7e654e33443ee90fafb5810b164ebe3b6d
Nov 03 16:40:24.620904 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.620872    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl" event={"ID":"8ea72c89-5ec4-4822-be54-be0b1288d62c","Type":"ContainerStarted","Data":"d377dd7cd62fc77485a113deb1de9f7e654e33443ee90fafb5810b164ebe3b6d"}
Nov 03 16:40:24.738515 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:24.738480    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"]
Nov 03 16:40:24.739630 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:40:24.739603    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e4bfb6_f892_4dae_8e86_36af9c6def94.slice/crio-a285f2409f31d5cbb2f6250b336f7665851e0cf39f2692d7b1153de3e8dd21a9 WatchSource:0}: Error finding container a285f2409f31d5cbb2f6250b336f7665851e0cf39f2692d7b1153de3e8dd21a9: Status 404 returned error can't find the container with id a285f2409f31d5cbb2f6250b336f7665851e0cf39f2692d7b1153de3e8dd21a9
Nov 03 16:40:25.628439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:25.628397    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4" event={"ID":"c2e4bfb6-f892-4dae-8e86-36af9c6def94","Type":"ContainerStarted","Data":"da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c"}
Nov 03 16:40:25.628931 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:25.628464    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4" event={"ID":"c2e4bfb6-f892-4dae-8e86-36af9c6def94","Type":"ContainerStarted","Data":"a285f2409f31d5cbb2f6250b336f7665851e0cf39f2692d7b1153de3e8dd21a9"}
Nov 03 16:40:25.628931 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:25.628496    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:40:25.630173 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:25.630148    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl" event={"ID":"8ea72c89-5ec4-4822-be54-be0b1288d62c","Type":"ContainerStarted","Data":"caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8"}
Nov 03 16:40:25.630300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:25.630196    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:40:25.761237 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:25.761184    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4" podStartSLOduration=2.761167581 podStartE2EDuration="2.761167581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:40:25.757395856 +0000 UTC m=+1711.581847131" watchObservedRunningTime="2026-11-03 16:40:25.761167581 +0000 UTC m=+1711.585619049"
Nov 03 16:40:25.908833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:25.907974    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl" podStartSLOduration=2.907951437 podStartE2EDuration="2.907951437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:40:25.889271446 +0000 UTC m=+1711.713722723" watchObservedRunningTime="2026-11-03 16:40:25.907951437 +0000 UTC m=+1711.732402709"
Nov 03 16:40:33.414898 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:40:33.414855    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:40:36.640312 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:36.640277    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:40:36.640778 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:36.640461    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:40:39.421286 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.421198    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 03 16:40:39.421286 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.421266    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 03 16:40:39.449684 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.449644    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"]
Nov 03 16:40:39.453017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.452992    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.513764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.511371    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ba93ef4-459e-40b1-865d-fa5234b41a10-istiod-ca-cert\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.513764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.511452    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-certs\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.513764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.511498    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-podinfo\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.513764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.511525    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-envoy\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.513764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.511597    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9hxf\" (UniqueName: \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-kube-api-access-c9hxf\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.513764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.511626    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-data\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.513764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.511682    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-socket\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.513764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.511795    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-credential-socket\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.513764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.511837    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-token\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.546761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.544542    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-23c440-openshift-default-dockercfg-z48tf\""
Nov 03 16:40:39.614788 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.601348    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"]
Nov 03 16:40:39.621754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.619018    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-credential-socket\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.621754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.619092    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-token\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.621754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.619175    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ba93ef4-459e-40b1-865d-fa5234b41a10-istiod-ca-cert\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.621754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.619226    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-certs\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.621754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.619267    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-podinfo\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.621754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.619293    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-envoy\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.621754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.619363    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9hxf\" (UniqueName: \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-kube-api-access-c9hxf\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.621754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.619389    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-data\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.621754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.619446    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-socket\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.668386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.629114    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-socket\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.668386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.629392    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-credential-socket\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.668386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.630354    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ba93ef4-459e-40b1-865d-fa5234b41a10-istiod-ca-cert\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.668386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.630592    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-certs\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.674618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.674524    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-data\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.701757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.688573    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-podinfo\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.701757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.690875    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-envoy\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.701757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.700885    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9hxf\" (UniqueName: \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-kube-api-access-c9hxf\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.710788 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.709784    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-token\") pod \"e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") " pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.772141 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.772106    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:39.774602 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.774559    5605 generic.go:356] "Generic (PLEG): container finished" podID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerID="1a56961bfa843ddf73915395d367b9cda57b21030184ddfc61d097f2a3fdc630" exitCode=0
Nov 03 16:40:39.774911 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.774876    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" event={"ID":"84b165c2-3d89-452c-852b-ffa8ca079c72","Type":"ContainerDied","Data":"1a56961bfa843ddf73915395d367b9cda57b21030184ddfc61d097f2a3fdc630"}
Nov 03 16:40:39.775521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.775320    5605 scope.go:122] "RemoveContainer" containerID="1a56961bfa843ddf73915395d367b9cda57b21030184ddfc61d097f2a3fdc630"
Nov 03 16:40:39.781966 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.781924    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:39.782115 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.782025    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:39.782115 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:39.782081    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:40.253120 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:40.253000    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"]
Nov 03 16:40:40.253572 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:40:40.253382    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba93ef4_459e_40b1_865d_fa5234b41a10.slice/crio-c53b0e811446c2af3dad224adcf95397903e04efa84ba7abb32b110403ac5110 WatchSource:0}: Error finding container c53b0e811446c2af3dad224adcf95397903e04efa84ba7abb32b110403ac5110: Status 404 returned error can't find the container with id c53b0e811446c2af3dad224adcf95397903e04efa84ba7abb32b110403ac5110
Nov 03 16:40:40.257027 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:40.256981    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:40.257134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:40.257074    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:40.257134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:40.257126    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:40.784640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:40.784595    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" event={"ID":"84b165c2-3d89-452c-852b-ffa8ca079c72","Type":"ContainerStarted","Data":"eb83b42d19b9983fcd1444a1b0d1178fd5a9c669fbae84ef7e4e6dee60b63e16"}
Nov 03 16:40:40.787108 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:40.787073    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" event={"ID":"5ba93ef4-459e-40b1-865d-fa5234b41a10","Type":"ContainerStarted","Data":"079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13"}
Nov 03 16:40:40.787302 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:40.787284    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" event={"ID":"5ba93ef4-459e-40b1-865d-fa5234b41a10","Type":"ContainerStarted","Data":"c53b0e811446c2af3dad224adcf95397903e04efa84ba7abb32b110403ac5110"}
Nov 03 16:40:41.036452 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:41.036351    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" podStartSLOduration=2.036335448 podStartE2EDuration="2.036335448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:40:41.034299916 +0000 UTC m=+1726.858751189" watchObservedRunningTime="2026-11-03 16:40:41.036335448 +0000 UTC m=+1726.860786718"
Nov 03 16:40:41.538389 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:41.538342    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:40:41.538694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:41.538670    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 03 16:40:41.538802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:41.538720    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 03 16:40:41.772743 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:41.772695    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:41.778682 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:41.778647    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:41.809039 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:41.808952    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:41.810011 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:41.809989    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:40:42.538764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:42.538695    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 03 16:40:42.538951 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:42.538770    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 03 16:40:43.539102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:43.539058    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 03 16:40:43.539496 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:43.539123    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 03 16:40:44.538329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:44.538297    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 03 16:40:44.538503 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:44.538350    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 03 16:40:45.538934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.538898    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 03 16:40:45.539333 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.538965    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 03 16:40:45.907682 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.907647    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"]
Nov 03 16:40:45.920760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.914330    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:45.970925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.966686    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-socket\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:45.970925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.966794    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwf82\" (UniqueName: \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-kube-api-access-mwf82\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:45.970925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.966850    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-data\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:45.970925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.966930    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-certs\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:45.970925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.967014    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-podinfo\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:45.970925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.967056    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-envoy\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:45.970925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.967099    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c75fa68-0957-49ae-a013-0f8fae42dd31-istiod-ca-cert\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:45.970925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.967150    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-token\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:45.970925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.967239    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-credential-socket\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:45.972846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:45.972034    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-195095-openshift-default-dockercfg-b65sz\""
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.172160    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-socket\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.172229    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwf82\" (UniqueName: \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-kube-api-access-mwf82\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.172275    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-data\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.172333    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-certs\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.172397    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-podinfo\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.172430    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-envoy\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.172463    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c75fa68-0957-49ae-a013-0f8fae42dd31-istiod-ca-cert\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.172509    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-token\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.172576    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-credential-socket\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.173140    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-credential-socket\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.173834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.173597    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-socket\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.181692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.178638    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-envoy\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.181692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.179307    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c75fa68-0957-49ae-a013-0f8fae42dd31-istiod-ca-cert\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.181692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.179916    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-data\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.181692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.180162    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-certs\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.255834 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.255679    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"]
Nov 03 16:40:46.281874 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.271931    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-podinfo\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.314383 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.314298    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-token\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.315599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.315547    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwf82\" (UniqueName: \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-kube-api-access-mwf82\") pod \"e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") " pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:46.538450 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.538361    5605 patch_prober.go:29] interesting pod/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Nov 03 16:40:46.538450 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.538414    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Nov 03 16:40:46.569061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:46.569015    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:47.025554 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:47.025524    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"]
Nov 03 16:40:47.027243 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:40:47.027209    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c75fa68_0957_49ae_a013_0f8fae42dd31.slice/crio-c1d51d236de0201737b4bb43d2ce1d9b3f0bc9633769a9c7c9dd4819bd6aa067 WatchSource:0}: Error finding container c1d51d236de0201737b4bb43d2ce1d9b3f0bc9633769a9c7c9dd4819bd6aa067: Status 404 returned error can't find the container with id c1d51d236de0201737b4bb43d2ce1d9b3f0bc9633769a9c7c9dd4819bd6aa067
Nov 03 16:40:47.029793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:47.029751    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:47.029865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:47.029828    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:47.029914 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:47.029861    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:40:47.543304 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:47.543269    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:40:47.543770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:47.543753    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:40:47.544401 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:47.544384    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:40:47.849957 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:47.849917    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" event={"ID":"0c75fa68-0957-49ae-a013-0f8fae42dd31","Type":"ContainerStarted","Data":"5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c"}
Nov 03 16:40:47.850420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:47.849990    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" event={"ID":"0c75fa68-0957-49ae-a013-0f8fae42dd31","Type":"ContainerStarted","Data":"c1d51d236de0201737b4bb43d2ce1d9b3f0bc9633769a9c7c9dd4819bd6aa067"}
Nov 03 16:40:47.975987 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:47.975933    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" podStartSLOduration=2.975917174 podStartE2EDuration="2.975917174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:40:47.97460031 +0000 UTC m=+1733.799051581" watchObservedRunningTime="2026-11-03 16:40:47.975917174 +0000 UTC m=+1733.800368495"
Nov 03 16:40:48.569525 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:48.569495    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:48.575460 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:48.575438    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:48.856563 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:48.856532    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:48.857541 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:48.857521    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:40:48.976517 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:48.976484    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"]
Nov 03 16:40:48.979892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:48.979866    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.030824 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.030792    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-maas-api-tls\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.031129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.030881    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b3dae34-f264-44fd-8aae-725e3905595a-openshift-service-ca\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.031542 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.031509    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-metrics-certs\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.031659 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.031628    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbc25\" (UniqueName: \"kubernetes.io/projected/8b3dae34-f264-44fd-8aae-725e3905595a-kube-api-access-tbc25\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.038389 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.038342    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-cc4dcd-serving-cert\""
Nov 03 16:40:49.040431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.040382    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"]
Nov 03 16:40:49.132995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.132918    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-maas-api-tls\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.132995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.132979    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b3dae34-f264-44fd-8aae-725e3905595a-openshift-service-ca\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.133177 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:40:49.133083    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-serving-cert: secret "maas-api-e2e-api-a-cc4dcd-serving-cert" not found
Nov 03 16:40:49.133177 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.133102    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-metrics-certs\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.133177 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:40:49.133162    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-maas-api-tls podName:8b3dae34-f264-44fd-8aae-725e3905595a nodeName:}" failed. No retries permitted until 2026-11-03 16:40:49.633142162 +0000 UTC m=+1735.457593427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-maas-api-tls") pod "maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs" (UID: "8b3dae34-f264-44fd-8aae-725e3905595a") : secret "maas-api-e2e-api-a-cc4dcd-serving-cert" not found
Nov 03 16:40:49.133309 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.133184    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbc25\" (UniqueName: \"kubernetes.io/projected/8b3dae34-f264-44fd-8aae-725e3905595a-kube-api-access-tbc25\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.133612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.133583    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b3dae34-f264-44fd-8aae-725e3905595a-openshift-service-ca\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.135568 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.135551    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-metrics-certs\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.194930 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.194903    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbc25\" (UniqueName: \"kubernetes.io/projected/8b3dae34-f264-44fd-8aae-725e3905595a-kube-api-access-tbc25\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.640966 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.640932    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-maas-api-tls\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.643505 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.643486    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-maas-api-tls\") pod \"maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:49.893618 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:49.893518    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:50.241817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.241783    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"]
Nov 03 16:40:50.271798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.264058    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:40:50.287420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.284611    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"]
Nov 03 16:40:50.335073 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.330993    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-23c440\""
Nov 03 16:40:50.335073 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.331164    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-23c440-dockercfg-bnvsh\""
Nov 03 16:40:50.422808 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.418674    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/978db8fe-4aa9-40a6-8183-def3b49e2b68-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt\" (UID: \"978db8fe-4aa9-40a6-8183-def3b49e2b68\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:40:50.422808 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.419167    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dp9m\" (UniqueName: \"kubernetes.io/projected/978db8fe-4aa9-40a6-8183-def3b49e2b68-kube-api-access-7dp9m\") pod \"payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt\" (UID: \"978db8fe-4aa9-40a6-8183-def3b49e2b68\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:40:50.465754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.463812    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"]
Nov 03 16:40:50.545057 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.545006    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/978db8fe-4aa9-40a6-8183-def3b49e2b68-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt\" (UID: \"978db8fe-4aa9-40a6-8183-def3b49e2b68\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:40:50.548365 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.545467    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:40:50.548676 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.546313    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/978db8fe-4aa9-40a6-8183-def3b49e2b68-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt\" (UID: \"978db8fe-4aa9-40a6-8183-def3b49e2b68\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:40:50.548953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.548240    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dp9m\" (UniqueName: \"kubernetes.io/projected/978db8fe-4aa9-40a6-8183-def3b49e2b68-kube-api-access-7dp9m\") pod \"payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt\" (UID: \"978db8fe-4aa9-40a6-8183-def3b49e2b68\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:40:50.551352 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.551323    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"]
Nov 03 16:40:50.607035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.607007    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"]
Nov 03 16:40:50.608652 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:40:50.608619    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b3dae34_f264_44fd_8aae_725e3905595a.slice/crio-66246be655caf57859d64ea7808c0dcf0afd4604c43c258afa081f0ac75750d5 WatchSource:0}: Error finding container 66246be655caf57859d64ea7808c0dcf0afd4604c43c258afa081f0ac75750d5: Status 404 returned error can't find the container with id 66246be655caf57859d64ea7808c0dcf0afd4604c43c258afa081f0ac75750d5
Nov 03 16:40:50.612552 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.612528    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dp9m\" (UniqueName: \"kubernetes.io/projected/978db8fe-4aa9-40a6-8183-def3b49e2b68-kube-api-access-7dp9m\") pod \"payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt\" (UID: \"978db8fe-4aa9-40a6-8183-def3b49e2b68\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:40:50.651377 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.651346    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eade339c-4ff4-49d3-b634-1ad4459904f3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-23c440-6576874c97-n966f\" (UID: \"eade339c-4ff4-49d3-b634-1ad4459904f3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:40:50.651527 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.651479    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tb9s\" (UniqueName: \"kubernetes.io/projected/eade339c-4ff4-49d3-b634-1ad4459904f3-kube-api-access-2tb9s\") pod \"payload-processing-e2e-inf-a-23c440-6576874c97-n966f\" (UID: \"eade339c-4ff4-49d3-b634-1ad4459904f3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:40:50.752500 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.752390    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eade339c-4ff4-49d3-b634-1ad4459904f3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-23c440-6576874c97-n966f\" (UID: \"eade339c-4ff4-49d3-b634-1ad4459904f3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:40:50.752683 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.752527    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tb9s\" (UniqueName: \"kubernetes.io/projected/eade339c-4ff4-49d3-b634-1ad4459904f3-kube-api-access-2tb9s\") pod \"payload-processing-e2e-inf-a-23c440-6576874c97-n966f\" (UID: \"eade339c-4ff4-49d3-b634-1ad4459904f3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:40:50.753209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.753186    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eade339c-4ff4-49d3-b634-1ad4459904f3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-23c440-6576874c97-n966f\" (UID: \"eade339c-4ff4-49d3-b634-1ad4459904f3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:40:50.813145 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.813103    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tb9s\" (UniqueName: \"kubernetes.io/projected/eade339c-4ff4-49d3-b634-1ad4459904f3-kube-api-access-2tb9s\") pod \"payload-processing-e2e-inf-a-23c440-6576874c97-n966f\" (UID: \"eade339c-4ff4-49d3-b634-1ad4459904f3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:40:50.868586 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.868551    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:40:50.870154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.870126    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs" event={"ID":"8b3dae34-f264-44fd-8aae-725e3905595a","Type":"ContainerStarted","Data":"66246be655caf57859d64ea7808c0dcf0afd4604c43c258afa081f0ac75750d5"}
Nov 03 16:40:50.891741 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:50.891689    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:40:51.251517 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:51.251488    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"]
Nov 03 16:40:51.291975 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:40:51.291947    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeade339c_4ff4_49d3_b634_1ad4459904f3.slice/crio-7f937d05323efdad7076878b981ed88ce18727aa87dc0b4de873967a9f5a8921 WatchSource:0}: Error finding container 7f937d05323efdad7076878b981ed88ce18727aa87dc0b4de873967a9f5a8921: Status 404 returned error can't find the container with id 7f937d05323efdad7076878b981ed88ce18727aa87dc0b4de873967a9f5a8921
Nov 03 16:40:51.303401 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:51.303375    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"]
Nov 03 16:40:51.307712 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:40:51.307685    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod978db8fe_4aa9_40a6_8183_def3b49e2b68.slice/crio-fc25b3f23e098764b57900eba3f6af429c9ee7968dd059506fe884ae38f8b179 WatchSource:0}: Error finding container fc25b3f23e098764b57900eba3f6af429c9ee7968dd059506fe884ae38f8b179: Status 404 returned error can't find the container with id fc25b3f23e098764b57900eba3f6af429c9ee7968dd059506fe884ae38f8b179
Nov 03 16:40:51.877974 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:51.877882    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs" event={"ID":"8b3dae34-f264-44fd-8aae-725e3905595a","Type":"ContainerStarted","Data":"978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d"}
Nov 03 16:40:51.878153 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:51.877990    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:51.879699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:51.879669    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt" event={"ID":"978db8fe-4aa9-40a6-8183-def3b49e2b68","Type":"ContainerStarted","Data":"7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373"}
Nov 03 16:40:51.879863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:51.879709    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt" event={"ID":"978db8fe-4aa9-40a6-8183-def3b49e2b68","Type":"ContainerStarted","Data":"fc25b3f23e098764b57900eba3f6af429c9ee7968dd059506fe884ae38f8b179"}
Nov 03 16:40:51.879863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:51.879748    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:40:51.881349 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:51.881322    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f" event={"ID":"eade339c-4ff4-49d3-b634-1ad4459904f3","Type":"ContainerStarted","Data":"3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5"}
Nov 03 16:40:51.881485 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:51.881362    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f" event={"ID":"eade339c-4ff4-49d3-b634-1ad4459904f3","Type":"ContainerStarted","Data":"7f937d05323efdad7076878b981ed88ce18727aa87dc0b4de873967a9f5a8921"}
Nov 03 16:40:51.881485 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:51.881394    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:40:52.001740 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:52.001677    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs" podStartSLOduration=3.024746909 podStartE2EDuration="4.001657952s" totalImagesPullingTime="976.911043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:40:51.999421076 +0000 UTC m=+1737.823872345" watchObservedRunningTime="2026-11-03 16:40:52.001657952 +0000 UTC m=+1737.826109228"
Nov 03 16:40:52.122278 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:52.122226    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt" podStartSLOduration=2.12221182 podStartE2EDuration="2.12221182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:40:52.120983331 +0000 UTC m=+1737.945434599" watchObservedRunningTime="2026-11-03 16:40:52.12221182 +0000 UTC m=+1737.946663090"
Nov 03 16:40:52.244193 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:52.244083    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f" podStartSLOduration=2.244066878 podStartE2EDuration="2.244066878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:40:52.243341133 +0000 UTC m=+1738.067792406" watchObservedRunningTime="2026-11-03 16:40:52.244066878 +0000 UTC m=+1738.068518152"
Nov 03 16:40:57.790716 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:57.785425    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"]
Nov 03 16:40:57.790716 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:57.790758    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 16:40:57.848010 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:57.847962    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"]
Nov 03 16:40:57.912948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:57.911761    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-195095\""
Nov 03 16:40:57.912948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:57.911996    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-195095-dockercfg-pw698\""
Nov 03 16:40:57.986855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:57.928019    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddnd4\" (UniqueName: \"kubernetes.io/projected/626e2175-2d9f-483c-92bd-f3538bd95f22-kube-api-access-ddnd4\") pod \"payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j\" (UID: \"626e2175-2d9f-483c-92bd-f3538bd95f22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 16:40:57.986855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:57.982877    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/626e2175-2d9f-483c-92bd-f3538bd95f22-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j\" (UID: \"626e2175-2d9f-483c-92bd-f3538bd95f22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 16:40:57.986855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:57.978945    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:40:58.029826 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.029787    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"]
Nov 03 16:40:58.053397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.053312    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 16:40:58.079960 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.079931    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"]
Nov 03 16:40:58.086462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.086432    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddnd4\" (UniqueName: \"kubernetes.io/projected/626e2175-2d9f-483c-92bd-f3538bd95f22-kube-api-access-ddnd4\") pod \"payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j\" (UID: \"626e2175-2d9f-483c-92bd-f3538bd95f22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 16:40:58.086962 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.086937    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/626e2175-2d9f-483c-92bd-f3538bd95f22-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j\" (UID: \"626e2175-2d9f-483c-92bd-f3538bd95f22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 16:40:58.088699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.088507    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/626e2175-2d9f-483c-92bd-f3538bd95f22-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j\" (UID: \"626e2175-2d9f-483c-92bd-f3538bd95f22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 16:40:58.152784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.148953    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddnd4\" (UniqueName: \"kubernetes.io/projected/626e2175-2d9f-483c-92bd-f3538bd95f22-kube-api-access-ddnd4\") pod \"payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j\" (UID: \"626e2175-2d9f-483c-92bd-f3538bd95f22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 16:40:58.188529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.188493    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g882r\" (UniqueName: \"kubernetes.io/projected/b058295c-3fa6-409d-83ee-981d4ee60381-kube-api-access-g882r\") pod \"payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl\" (UID: \"b058295c-3fa6-409d-83ee-981d4ee60381\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 16:40:58.188777 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.188753    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b058295c-3fa6-409d-83ee-981d4ee60381-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl\" (UID: \"b058295c-3fa6-409d-83ee-981d4ee60381\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 16:40:58.290429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.290393    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b058295c-3fa6-409d-83ee-981d4ee60381-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl\" (UID: \"b058295c-3fa6-409d-83ee-981d4ee60381\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 16:40:58.290623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.290505    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g882r\" (UniqueName: \"kubernetes.io/projected/b058295c-3fa6-409d-83ee-981d4ee60381-kube-api-access-g882r\") pod \"payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl\" (UID: \"b058295c-3fa6-409d-83ee-981d4ee60381\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 16:40:58.291245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.291224    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b058295c-3fa6-409d-83ee-981d4ee60381-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl\" (UID: \"b058295c-3fa6-409d-83ee-981d4ee60381\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 16:40:58.351354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.351325    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g882r\" (UniqueName: \"kubernetes.io/projected/b058295c-3fa6-409d-83ee-981d4ee60381-kube-api-access-g882r\") pod \"payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl\" (UID: \"b058295c-3fa6-409d-83ee-981d4ee60381\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 16:40:58.386882 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.386844    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 16:40:58.422172 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.422145    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 16:40:58.776138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.776049    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"]
Nov 03 16:40:58.815323 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:40:58.815287    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb058295c_3fa6_409d_83ee_981d4ee60381.slice/crio-c421d40657d6dfafe91a2ad432d79dfed3a9c146fafa0901deaed7b217725159 WatchSource:0}: Error finding container c421d40657d6dfafe91a2ad432d79dfed3a9c146fafa0901deaed7b217725159: Status 404 returned error can't find the container with id c421d40657d6dfafe91a2ad432d79dfed3a9c146fafa0901deaed7b217725159
Nov 03 16:40:58.833109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:58.833076    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"]
Nov 03 16:40:58.834030 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:40:58.834002    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod626e2175_2d9f_483c_92bd_f3538bd95f22.slice/crio-abbbccedd05f03c8610dfadaf0dc963de9e5d8eca610f5b9bd39dc4c0b49d9c9 WatchSource:0}: Error finding container abbbccedd05f03c8610dfadaf0dc963de9e5d8eca610f5b9bd39dc4c0b49d9c9: Status 404 returned error can't find the container with id abbbccedd05f03c8610dfadaf0dc963de9e5d8eca610f5b9bd39dc4c0b49d9c9
Nov 03 16:40:59.050812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:59.047364    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl" event={"ID":"b058295c-3fa6-409d-83ee-981d4ee60381","Type":"ContainerStarted","Data":"5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e"}
Nov 03 16:40:59.050812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:59.047435    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl" event={"ID":"b058295c-3fa6-409d-83ee-981d4ee60381","Type":"ContainerStarted","Data":"c421d40657d6dfafe91a2ad432d79dfed3a9c146fafa0901deaed7b217725159"}
Nov 03 16:40:59.050812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:59.048543    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 16:40:59.057807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:59.055225    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j" event={"ID":"626e2175-2d9f-483c-92bd-f3538bd95f22","Type":"ContainerStarted","Data":"e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60"}
Nov 03 16:40:59.057807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:59.055284    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j" event={"ID":"626e2175-2d9f-483c-92bd-f3538bd95f22","Type":"ContainerStarted","Data":"abbbccedd05f03c8610dfadaf0dc963de9e5d8eca610f5b9bd39dc4c0b49d9c9"}
Nov 03 16:40:59.057807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:59.056110    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 16:40:59.171058 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:59.171009    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl" podStartSLOduration=2.17099504 podStartE2EDuration="2.17099504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:40:59.17043472 +0000 UTC m=+1744.994885994" watchObservedRunningTime="2026-11-03 16:40:59.17099504 +0000 UTC m=+1744.995446312"
Nov 03 16:40:59.329810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:40:59.312436    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j" podStartSLOduration=2.312413438 podStartE2EDuration="2.312413438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:40:59.294870944 +0000 UTC m=+1745.119322226" watchObservedRunningTime="2026-11-03 16:40:59.312413438 +0000 UTC m=+1745.136864710"
Nov 03 16:41:02.890700 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:02.890668    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:41:02.891158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:02.890774    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:41:03.393612 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:41:03.393575    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:41:05.365231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.365199    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"]
Nov 03 16:41:05.369585 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.369566    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.427991 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.427941    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-2537c1-openshift-default-dockercfg-jdfz9\""
Nov 03 16:41:05.460274 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.460238    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-credential-socket\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.460477 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.460283    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-certs\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.460477 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.460390    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-socket\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.460612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.460537    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-istio-token\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.460612 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.460568    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71409250-bb69-4586-8dc9-d22c888f1160-istio-podinfo\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.460690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.460644    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-data\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.460690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.460673    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpxkr\" (UniqueName: \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-kube-api-access-rpxkr\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.460798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.460785    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71409250-bb69-4586-8dc9-d22c888f1160-istiod-ca-cert\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.460849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.460835    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-envoy\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.481088 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.481056    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"]
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.563876    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpxkr\" (UniqueName: \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-kube-api-access-rpxkr\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.563997    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71409250-bb69-4586-8dc9-d22c888f1160-istiod-ca-cert\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.564058    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-envoy\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.564145    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-credential-socket\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.564173    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-certs\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.564204    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-socket\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.564329    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-istio-token\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.564416    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71409250-bb69-4586-8dc9-d22c888f1160-istio-podinfo\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.564508    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-data\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.565009    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-data\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.571169    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-certs\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.571293    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-socket\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.571416    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-credential-socket\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.573421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.571844    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71409250-bb69-4586-8dc9-d22c888f1160-istiod-ca-cert\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.601155 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.600677    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-envoy\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.601335 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.601197    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71409250-bb69-4586-8dc9-d22c888f1160-istio-podinfo\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.629045 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.628977    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpxkr\" (UniqueName: \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-kube-api-access-rpxkr\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.634383 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.634356    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-istio-token\") pod \"e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") " pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:05.686185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:05.686152    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:06.131199 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:06.131172    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"]
Nov 03 16:41:06.133002 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:41:06.132978    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71409250_bb69_4586_8dc9_d22c888f1160.slice/crio-26069e90ef3c9e25ea913dac08a8c3c5e22a47da7c337d4c28b052a7ca41520b WatchSource:0}: Error finding container 26069e90ef3c9e25ea913dac08a8c3c5e22a47da7c337d4c28b052a7ca41520b: Status 404 returned error can't find the container with id 26069e90ef3c9e25ea913dac08a8c3c5e22a47da7c337d4c28b052a7ca41520b
Nov 03 16:41:06.135776 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:06.135710    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:41:06.135856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:06.135813    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:41:06.135856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:06.135843    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:41:07.118153 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:07.118121    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" event={"ID":"71409250-bb69-4586-8dc9-d22c888f1160","Type":"ContainerStarted","Data":"eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b"}
Nov 03 16:41:07.118560 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:07.118202    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" event={"ID":"71409250-bb69-4586-8dc9-d22c888f1160","Type":"ContainerStarted","Data":"26069e90ef3c9e25ea913dac08a8c3c5e22a47da7c337d4c28b052a7ca41520b"}
Nov 03 16:41:07.355851 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:07.355798    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" podStartSLOduration=2.355779274 podStartE2EDuration="2.355779274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:41:07.301670639 +0000 UTC m=+1753.126121909" watchObservedRunningTime="2026-11-03 16:41:07.355779274 +0000 UTC m=+1753.180230558"
Nov 03 16:41:07.686552 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:07.686509    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:07.692201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:07.692178    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:08.124678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:08.124640    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:08.125540 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:08.125520    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:41:11.134788 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:11.133967    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 16:41:11.164545 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:11.163910    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 16:41:14.021097 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.021054    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"]
Nov 03 16:41:14.038756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.034203    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:41:14.052773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.044195    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9\" (UID: \"54734e4d-d0a1-4a3a-a617-fbffc691d3e5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:41:14.052773 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.044253    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x88ll\" (UniqueName: \"kubernetes.io/projected/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-kube-api-access-x88ll\") pod \"payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9\" (UID: \"54734e4d-d0a1-4a3a-a617-fbffc691d3e5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:41:14.100767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.089833    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"]
Nov 03 16:41:14.278415 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.278332    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-2537c1-dockercfg-vz9zf\""
Nov 03 16:41:14.278712 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.278695    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-2537c1\""
Nov 03 16:41:14.299868 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.299422    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9\" (UID: \"54734e4d-d0a1-4a3a-a617-fbffc691d3e5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:41:14.300774 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.300558    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x88ll\" (UniqueName: \"kubernetes.io/projected/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-kube-api-access-x88ll\") pod \"payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9\" (UID: \"54734e4d-d0a1-4a3a-a617-fbffc691d3e5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:41:14.355922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.351833    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9\" (UID: \"54734e4d-d0a1-4a3a-a617-fbffc691d3e5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:41:14.379185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.379147    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x88ll\" (UniqueName: \"kubernetes.io/projected/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-kube-api-access-x88ll\") pod \"payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9\" (UID: \"54734e4d-d0a1-4a3a-a617-fbffc691d3e5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:41:14.430016 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.416717    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"]
Nov 03 16:41:14.442739 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.442688    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"]
Nov 03 16:41:14.442975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.442959    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:41:14.527852 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.527820    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5s7l\" (UniqueName: \"kubernetes.io/projected/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-kube-api-access-r5s7l\") pod \"payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst\" (UID: \"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:41:14.528089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.528064    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst\" (UID: \"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:41:14.615190 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.615159    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:41:14.630131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.630102    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5s7l\" (UniqueName: \"kubernetes.io/projected/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-kube-api-access-r5s7l\") pod \"payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst\" (UID: \"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:41:14.630392 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.630372    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst\" (UID: \"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:41:14.631031 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.631011    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst\" (UID: \"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:41:14.727814 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.726136    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5s7l\" (UniqueName: \"kubernetes.io/projected/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-kube-api-access-r5s7l\") pod \"payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst\" (UID: \"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:41:14.758651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:14.758352    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:41:15.021944 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:15.021917    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"]
Nov 03 16:41:15.022303 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:41:15.022276    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54734e4d_d0a1_4a3a_a617_fbffc691d3e5.slice/crio-4b5a934f9088f3a6f0ca3e2a61ce7a55396f22aa1dcaa2a1b97fd4305d299861 WatchSource:0}: Error finding container 4b5a934f9088f3a6f0ca3e2a61ce7a55396f22aa1dcaa2a1b97fd4305d299861: Status 404 returned error can't find the container with id 4b5a934f9088f3a6f0ca3e2a61ce7a55396f22aa1dcaa2a1b97fd4305d299861
Nov 03 16:41:15.210348 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:15.210318    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"]
Nov 03 16:41:15.210959 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:41:15.210932    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod814d68eb_8cb5_4eea_8c2a_006a6e8b80e7.slice/crio-9fd2239c96dcd2b44484246071b5d4c90d141e3f2846918633a87fb213e83a22 WatchSource:0}: Error finding container 9fd2239c96dcd2b44484246071b5d4c90d141e3f2846918633a87fb213e83a22: Status 404 returned error can't find the container with id 9fd2239c96dcd2b44484246071b5d4c90d141e3f2846918633a87fb213e83a22
Nov 03 16:41:15.414772 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:15.414719    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst" event={"ID":"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7","Type":"ContainerStarted","Data":"af5f542bf6b808a100df23796a0fcd2a746b2e57a52233fca64abff48416562c"}
Nov 03 16:41:15.414958 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:15.414801    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst" event={"ID":"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7","Type":"ContainerStarted","Data":"9fd2239c96dcd2b44484246071b5d4c90d141e3f2846918633a87fb213e83a22"}
Nov 03 16:41:15.416530 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:15.416505    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9" event={"ID":"54734e4d-d0a1-4a3a-a617-fbffc691d3e5","Type":"ContainerStarted","Data":"0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e"}
Nov 03 16:41:15.416664 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:15.416545    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9" event={"ID":"54734e4d-d0a1-4a3a-a617-fbffc691d3e5","Type":"ContainerStarted","Data":"4b5a934f9088f3a6f0ca3e2a61ce7a55396f22aa1dcaa2a1b97fd4305d299861"}
Nov 03 16:41:15.416664 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:15.416576    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:41:15.537265 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:15.537152    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst" podStartSLOduration=1.537137703 podStartE2EDuration="1.537137703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:41:15.536860615 +0000 UTC m=+1761.361311885" watchObservedRunningTime="2026-11-03 16:41:15.537137703 +0000 UTC m=+1761.361589004"
Nov 03 16:41:15.683754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:15.680314    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9" podStartSLOduration=2.6802908800000003 podStartE2EDuration="2.68029088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:41:15.679026157 +0000 UTC m=+1761.503477430" watchObservedRunningTime="2026-11-03 16:41:15.68029088 +0000 UTC m=+1761.504742150"
Nov 03 16:41:16.423298 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:16.423266    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:41:22.468148 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:22.468115    5605 generic.go:356] "Generic (PLEG): container finished" podID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerID="fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8" exitCode=0
Nov 03 16:41:22.468607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:22.468191    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" event={"ID":"423c492d-e0e6-442f-949d-c4cee39a9d04","Type":"ContainerDied","Data":"fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8"}
Nov 03 16:41:22.468710 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:22.468695    5605 scope.go:122] "RemoveContainer" containerID="fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8"
Nov 03 16:41:22.470445 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:22.470411    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:41:22.470562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:22.470503    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:41:22.470562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:22.470555    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:41:23.476947 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:23.476913    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" event={"ID":"423c492d-e0e6-442f-949d-c4cee39a9d04","Type":"ContainerStarted","Data":"f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb"}
Nov 03 16:41:23.558405 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:23.558374    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:41:23.559522 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:23.559494    5605 patch_prober.go:29] interesting pod/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 03 16:41:23.559662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:23.559544    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 03 16:41:24.559227 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:24.559190    5605 patch_prober.go:29] interesting pod/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 03 16:41:24.559649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:24.559241    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 03 16:41:25.121530 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:25.121491    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:41:25.558799 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:25.558670    5605 patch_prober.go:29] interesting pod/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 03 16:41:25.558799 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:25.558775    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 03 16:41:26.426598 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:26.426567    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:41:26.559114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:26.559080    5605 patch_prober.go:29] interesting pod/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 03 16:41:26.559295 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:26.559143    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 03 16:41:27.432802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:27.432767    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:41:27.558905 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:27.558869    5605 patch_prober.go:29] interesting pod/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 03 16:41:27.559170 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:27.559143    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 03 16:41:28.559327 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:28.559293    5605 patch_prober.go:29] interesting pod/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 03 16:41:28.559766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:28.559357    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 03 16:41:29.563870 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.563837    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:41:29.564918 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.564897    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:41:29.699908 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.699866    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"]
Nov 03 16:41:29.703556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.703530    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:29.761094 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.761057    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"]
Nov 03 16:41:29.790931 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.790893    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-cb494a-serving-cert\""
Nov 03 16:41:29.830708 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.830619    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4srx5\" (UniqueName: \"kubernetes.io/projected/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-kube-api-access-4srx5\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:29.830880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.830835    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-metrics-certs\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:29.831121 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.831096    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-maas-api-tls\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:29.831290 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.831149    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-openshift-service-ca\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:29.951954 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.951906    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-maas-api-tls\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:29.972950 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.972900    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-openshift-service-ca\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:29.974304 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.974274    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4srx5\" (UniqueName: \"kubernetes.io/projected/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-kube-api-access-4srx5\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:29.975163 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.975136    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-metrics-certs\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:29.975721 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:41:29.955946    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-serving-cert: secret "maas-api-e2e-api-a-cb494a-serving-cert" not found
Nov 03 16:41:29.975990 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:41:29.975974    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-maas-api-tls podName:d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb nodeName:}" failed. No retries permitted until 2026-11-03 16:41:30.475947671 +0000 UTC m=+1776.300398921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-maas-api-tls") pod "maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd" (UID: "d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb") : secret "maas-api-e2e-api-a-cb494a-serving-cert" not found
Nov 03 16:41:29.976922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.974031    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-openshift-service-ca\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:29.996636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:29.990115    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-metrics-certs\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:30.037141 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:30.037109    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4srx5\" (UniqueName: \"kubernetes.io/projected/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-kube-api-access-4srx5\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:30.487661 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:30.487631    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-maas-api-tls\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:30.490223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:30.490187    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-maas-api-tls\") pod \"maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:30.625934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:30.625894    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:31.015835 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:41:31.015788    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd32e79ed_90dd_402c_88a3_5e0ed5ffe2eb.slice/crio-3268bdbf5dbd47c69bd0bb6dd3ddc787ab18905b6fa7d9559be2ac2d2327e78a WatchSource:0}: Error finding container 3268bdbf5dbd47c69bd0bb6dd3ddc787ab18905b6fa7d9559be2ac2d2327e78a: Status 404 returned error can't find the container with id 3268bdbf5dbd47c69bd0bb6dd3ddc787ab18905b6fa7d9559be2ac2d2327e78a
Nov 03 16:41:31.016309 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:31.016288    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"]
Nov 03 16:41:31.557397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:31.557359    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd" event={"ID":"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb","Type":"ContainerStarted","Data":"3268bdbf5dbd47c69bd0bb6dd3ddc787ab18905b6fa7d9559be2ac2d2327e78a"}
Nov 03 16:41:32.565765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:32.565714    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd" event={"ID":"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb","Type":"ContainerStarted","Data":"00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94"}
Nov 03 16:41:32.566241 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:32.565798    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:32.688332 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:32.688270    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd" podStartSLOduration=2.820408562 podStartE2EDuration="3.688254471s" totalImagesPullingTime="867.845909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:41:32.687366639 +0000 UTC m=+1778.511817934" watchObservedRunningTime="2026-11-03 16:41:32.688254471 +0000 UTC m=+1778.512705788"
Nov 03 16:41:33.393791 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:41:33.393755    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:41:38.578258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:38.578227    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:41:54.552508 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:54.552474    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:41:54.558110 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:54.558084    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:41:58.446366 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.446329    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"]
Nov 03 16:41:58.452132 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.452112    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.508462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.508036    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-23c440-serving-cert\""
Nov 03 16:41:58.511803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.511771    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"]
Nov 03 16:41:58.610568 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.610533    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-maas-api-tls\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.610784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.610631    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g67d8\" (UniqueName: \"kubernetes.io/projected/c62b1d42-affc-4890-b8fa-7d044b2bba96-kube-api-access-g67d8\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.610784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.610684    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c62b1d42-affc-4890-b8fa-7d044b2bba96-openshift-service-ca\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.610922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.610860    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-metrics-certs\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.713255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.713151    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g67d8\" (UniqueName: \"kubernetes.io/projected/c62b1d42-affc-4890-b8fa-7d044b2bba96-kube-api-access-g67d8\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.713604 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.713570    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c62b1d42-affc-4890-b8fa-7d044b2bba96-openshift-service-ca\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.713917 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.713899    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-metrics-certs\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.714212 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.714191    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-maas-api-tls\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.725034 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.725004    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c62b1d42-affc-4890-b8fa-7d044b2bba96-openshift-service-ca\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.737380 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.737349    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-maas-api-tls\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.755335 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.755295    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-metrics-certs\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:58.777240 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:58.777167    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g67d8\" (UniqueName: \"kubernetes.io/projected/c62b1d42-affc-4890-b8fa-7d044b2bba96-kube-api-access-g67d8\") pod \"maas-api-e2e-inf-a-23c440-76fd77d444-454dn\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:59.066530 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:59.066495    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:41:59.454933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:59.454905    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"]
Nov 03 16:41:59.455461 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:41:59.455437    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc62b1d42_affc_4890_b8fa_7d044b2bba96.slice/crio-1bbd22910710a80482b87446303581ee037803d6af0992690236016a75492896 WatchSource:0}: Error finding container 1bbd22910710a80482b87446303581ee037803d6af0992690236016a75492896: Status 404 returned error can't find the container with id 1bbd22910710a80482b87446303581ee037803d6af0992690236016a75492896
Nov 03 16:41:59.456976 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:59.456956    5605 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 16:41:59.764719 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:41:59.764632    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn" event={"ID":"c62b1d42-affc-4890-b8fa-7d044b2bba96","Type":"ContainerStarted","Data":"1bbd22910710a80482b87446303581ee037803d6af0992690236016a75492896"}
Nov 03 16:42:00.774221 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:00.774123    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn" event={"ID":"c62b1d42-affc-4890-b8fa-7d044b2bba96","Type":"ContainerStarted","Data":"c499d9a432b020b64e2e853a2276c6ce0fc4f70f302b079429b6e61cbcbb18b5"}
Nov 03 16:42:00.774606 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:00.774223    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:42:00.898640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:00.898588    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn" podStartSLOduration=1.8831768549999999 podStartE2EDuration="2.898573687s" totalImagesPullingTime="1.015396832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:41:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:42:00.896584637 +0000 UTC m=+1806.721035907" watchObservedRunningTime="2026-11-03 16:42:00.898573687 +0000 UTC m=+1806.723025122"
Nov 03 16:42:02.917391 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:02.916816    5605 patch_prober.go:29] interesting pod/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": read tcp 10.133.0.2:40358->10.133.0.104:15021: read: connection reset by peer" start-of-body=
Nov 03 16:42:02.917391 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:02.916906    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": read tcp 10.133.0.2:40358->10.133.0.104:15021: read: connection reset by peer"
Nov 03 16:42:03.393558 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:42:03.393526    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:42:03.795125 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:03.795020    5605 generic.go:356] "Generic (PLEG): container finished" podID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerID="94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981" exitCode=0
Nov 03 16:42:03.795125 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:03.795108    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" event={"ID":"afb0f028-ecde-48ed-92b0-39276f7306e7","Type":"ContainerDied","Data":"94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981"}
Nov 03 16:42:03.795647 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:03.795629    5605 scope.go:122] "RemoveContainer" containerID="94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981"
Nov 03 16:42:03.797480 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:03.797449    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:42:03.797565 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:03.797519    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:42:03.797565 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:03.797553    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:42:04.804135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:04.804107    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" event={"ID":"afb0f028-ecde-48ed-92b0-39276f7306e7","Type":"ContainerStarted","Data":"d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18"}
Nov 03 16:42:05.200546 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:05.200514    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:42:05.200949 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:05.200863    5605 patch_prober.go:29] interesting pod/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 03 16:42:05.201276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:05.201247    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 03 16:42:06.200232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:06.200193    5605 patch_prober.go:29] interesting pod/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 03 16:42:06.200876 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:06.200843    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 03 16:42:06.809916 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:06.809884    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:42:07.200415 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:07.200380    5605 patch_prober.go:29] interesting pod/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 03 16:42:07.201076 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:07.201026    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 03 16:42:07.906488 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:07.906456    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"]
Nov 03 16:42:07.910461 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:07.910438    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:07.960966 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:07.960936    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"]
Nov 03 16:42:07.973458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:07.973428    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:07.973458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:07.973459    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9df6a16-4e47-4794-a2be-cd38093531bb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:07.973629 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:07.973564    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:07.973629 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:07.973591    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w22w\" (UniqueName: \"kubernetes.io/projected/b9df6a16-4e47-4794-a2be-cd38093531bb-kube-api-access-5w22w\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:08.008785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.008753    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-195095-serving-cert\""
Nov 03 16:42:08.074844 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.074814    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:08.075012 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.074855    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w22w\" (UniqueName: \"kubernetes.io/projected/b9df6a16-4e47-4794-a2be-cd38093531bb-kube-api-access-5w22w\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:08.075187 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.075158    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:08.075318 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.075220    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9df6a16-4e47-4794-a2be-cd38093531bb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:08.075318 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:42:08.075253    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-serving-cert: secret "maas-api-e2e-shared-b-w0-195095-serving-cert" not found
Nov 03 16:42:08.075318 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:42:08.075312    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-maas-api-tls podName:b9df6a16-4e47-4794-a2be-cd38093531bb nodeName:}" failed. No retries permitted until 2026-11-03 16:42:08.575295358 +0000 UTC m=+1814.399746626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-maas-api-tls") pod "maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8" (UID: "b9df6a16-4e47-4794-a2be-cd38093531bb") : secret "maas-api-e2e-shared-b-w0-195095-serving-cert" not found
Nov 03 16:42:08.075863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.075805    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9df6a16-4e47-4794-a2be-cd38093531bb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:08.077593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.077575    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:08.203981 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.203883    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w22w\" (UniqueName: \"kubernetes.io/projected/b9df6a16-4e47-4794-a2be-cd38093531bb-kube-api-access-5w22w\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:08.207686 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.207644    5605 patch_prober.go:29] interesting pod/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 03 16:42:08.214117 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.213975    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 03 16:42:08.598752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.591990    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:08.607755 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.606549    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:08.824146 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:08.824114    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:09.203819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:09.202599    5605 patch_prober.go:29] interesting pod/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 03 16:42:09.203819 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:09.202672    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 03 16:42:09.227575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:09.227547    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"]
Nov 03 16:42:09.228318 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:42:09.228289    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9df6a16_4e47_4794_a2be_cd38093531bb.slice/crio-4203a5471753ee5d952c47913445850b387255c00fe76dc02500d104e4b141e7 WatchSource:0}: Error finding container 4203a5471753ee5d952c47913445850b387255c00fe76dc02500d104e4b141e7: Status 404 returned error can't find the container with id 4203a5471753ee5d952c47913445850b387255c00fe76dc02500d104e4b141e7
Nov 03 16:42:09.842484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:09.842441    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8" event={"ID":"b9df6a16-4e47-4794-a2be-cd38093531bb","Type":"ContainerStarted","Data":"4203a5471753ee5d952c47913445850b387255c00fe76dc02500d104e4b141e7"}
Nov 03 16:42:10.206283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:10.206236    5605 patch_prober.go:29] interesting pod/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Nov 03 16:42:10.206484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:10.206318    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Nov 03 16:42:10.850830 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:10.850796    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8" event={"ID":"b9df6a16-4e47-4794-a2be-cd38093531bb","Type":"ContainerStarted","Data":"37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680"}
Nov 03 16:42:10.851238 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:10.850874    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:10.975077 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:10.975016    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8" podStartSLOduration=3.125071096 podStartE2EDuration="3.975001455s" totalImagesPullingTime="849.930359ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:42:10.972304476 +0000 UTC m=+1816.796755746" watchObservedRunningTime="2026-11-03 16:42:10.975001455 +0000 UTC m=+1816.799452771"
Nov 03 16:42:11.204839 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:11.204750    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:42:11.205178 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:11.205148    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:42:11.205833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:11.205816    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:42:16.863880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:16.863851    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 16:42:33.189784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.189677    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"]
Nov 03 16:42:33.195240 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.195219    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.216823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.216792    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjmkz\" (UniqueName: \"kubernetes.io/projected/f14c005c-83b1-4d64-9195-c52539ff7c35-kube-api-access-sjmkz\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.216948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.216911    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.217001 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.216972    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.217144 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.217130    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f14c005c-83b1-4d64-9195-c52539ff7c35-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.252165 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.252133    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"]
Nov 03 16:42:33.301157 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.301112    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-2537c1-serving-cert\""
Nov 03 16:42:33.335750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.332594    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f14c005c-83b1-4d64-9195-c52539ff7c35-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.353388 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.353355    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjmkz\" (UniqueName: \"kubernetes.io/projected/f14c005c-83b1-4d64-9195-c52539ff7c35-kube-api-access-sjmkz\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.485048 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.336769    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f14c005c-83b1-4d64-9195-c52539ff7c35-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.485573 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.485539    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.486962 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.486938    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.487745 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:42:33.486477    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-serving-cert: secret "maas-api-e2e-shared-b-w1-2537c1-serving-cert" not found
Nov 03 16:42:33.488622 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:42:33.488606    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-maas-api-tls podName:f14c005c-83b1-4d64-9195-c52539ff7c35 nodeName:}" failed. No retries permitted until 2026-11-03 16:42:33.988583968 +0000 UTC m=+1839.813035232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-maas-api-tls") pod "maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2" (UID: "f14c005c-83b1-4d64-9195-c52539ff7c35") : secret "maas-api-e2e-shared-b-w1-2537c1-serving-cert" not found
Nov 03 16:42:33.626750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.606422    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjmkz\" (UniqueName: \"kubernetes.io/projected/f14c005c-83b1-4d64-9195-c52539ff7c35-kube-api-access-sjmkz\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.657045 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:33.656886    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:33.671802 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:42:33.671755    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:42:34.057082 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:34.057051    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:34.060209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:34.060185    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:34.113515 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:34.111965    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:34.746703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:34.746615    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"]
Nov 03 16:42:34.750316 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:42:34.750288    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf14c005c_83b1_4d64_9195_c52539ff7c35.slice/crio-4aa202b2dc6a47ee64d87fe600786fa133899b51612ef9e608e21ab991202284 WatchSource:0}: Error finding container 4aa202b2dc6a47ee64d87fe600786fa133899b51612ef9e608e21ab991202284: Status 404 returned error can't find the container with id 4aa202b2dc6a47ee64d87fe600786fa133899b51612ef9e608e21ab991202284
Nov 03 16:42:35.068621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:35.068533    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2" event={"ID":"f14c005c-83b1-4d64-9195-c52539ff7c35","Type":"ContainerStarted","Data":"4aa202b2dc6a47ee64d87fe600786fa133899b51612ef9e608e21ab991202284"}
Nov 03 16:42:36.077264 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:36.077178    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2" event={"ID":"f14c005c-83b1-4d64-9195-c52539ff7c35","Type":"ContainerStarted","Data":"a0455e0331c2c9c94ff5de9b82706f034fdab0c86e10d29ac9e7ca17f222a893"}
Nov 03 16:42:36.077863 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:36.077845    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:36.227874 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:36.227807    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2" podStartSLOduration=2.306666146 podStartE2EDuration="3.227787409s" totalImagesPullingTime="921.121263ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:42:36.217176396 +0000 UTC m=+1842.041627670" watchObservedRunningTime="2026-11-03 16:42:36.227787409 +0000 UTC m=+1842.052238681"
Nov 03 16:42:42.850904 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:42.850859    5605 patch_prober.go:29] interesting pod/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:42:42.851444 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:42.850949    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:42:43.140572 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:43.140493    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:42:44.185549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:44.185508    5605 generic.go:356] "Generic (PLEG): container finished" podID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerID="079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13" exitCode=0
Nov 03 16:42:44.186056 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:44.185578    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" event={"ID":"5ba93ef4-459e-40b1-865d-fa5234b41a10","Type":"ContainerDied","Data":"079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13"}
Nov 03 16:42:44.186056 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:44.186019    5605 scope.go:122] "RemoveContainer" containerID="079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13"
Nov 03 16:42:44.187786 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:44.187755    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:42:44.187869 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:44.187830    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:42:44.187869 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:44.187864    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:42:45.213837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:45.213802    5605 generic.go:356] "Generic (PLEG): container finished" podID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerID="5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c" exitCode=0
Nov 03 16:42:45.214337 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:45.213874    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" event={"ID":"0c75fa68-0957-49ae-a013-0f8fae42dd31","Type":"ContainerDied","Data":"5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c"}
Nov 03 16:42:45.214422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:45.214404    5605 scope.go:122] "RemoveContainer" containerID="5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c"
Nov 03 16:42:45.216327 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:45.216303    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" event={"ID":"5ba93ef4-459e-40b1-865d-fa5234b41a10","Type":"ContainerStarted","Data":"6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6"}
Nov 03 16:42:45.216506 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:45.216479    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:42:45.216568 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:45.216553    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:42:45.216619 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:45.216594    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:42:45.772398 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:45.772367    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:42:45.772570 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:45.772506    5605 patch_prober.go:29] interesting pod/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Nov 03 16:42:45.772570 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:45.772554    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Nov 03 16:42:46.240955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:46.240923    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" event={"ID":"0c75fa68-0957-49ae-a013-0f8fae42dd31","Type":"ContainerStarted","Data":"c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66"}
Nov 03 16:42:46.569964 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:46.569874    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:42:46.571338 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:46.571311    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Nov 03 16:42:46.571493 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:46.571372    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Nov 03 16:42:46.773322 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:46.773286    5605 patch_prober.go:29] interesting pod/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Nov 03 16:42:46.773520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:46.773344    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Nov 03 16:42:47.570356 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:47.570322    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Nov 03 16:42:47.570899 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:47.570383    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Nov 03 16:42:47.773321 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:47.773284    5605 patch_prober.go:29] interesting pod/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Nov 03 16:42:47.773497 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:47.773359    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Nov 03 16:42:48.570124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:48.570087    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Nov 03 16:42:48.570319 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:48.570169    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Nov 03 16:42:48.773461 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:48.773423    5605 patch_prober.go:29] interesting pod/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Nov 03 16:42:48.773964 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:48.773485    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Nov 03 16:42:48.857440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:48.857412    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:42:49.571876 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:49.571834    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Nov 03 16:42:49.572047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:49.571898    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Nov 03 16:42:49.772764 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:49.772707    5605 patch_prober.go:29] interesting pod/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Nov 03 16:42:49.772934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:49.772792    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Nov 03 16:42:50.570389 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:50.570346    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Nov 03 16:42:50.570933 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:50.570407    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Nov 03 16:42:50.777245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:50.777213    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:42:50.777640 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:50.777620    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:42:50.778423 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:50.778403    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:42:51.588747 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:51.588048    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:42:51.589595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:42:51.589568    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 16:43:03.396994 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:43:03.396959    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:43:14.580244 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.579925    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"]
Nov 03 16:43:14.587873 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.587847    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:43:14.617845 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.617794    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a89c7b2-b9d2-4105-be8a-07232645205d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-66f145-b99dc85-bzg66\" (UID: \"0a89c7b2-b9d2-4105-be8a-07232645205d\") " pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:43:14.618235 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.618207    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8fx6\" (UniqueName: \"kubernetes.io/projected/0a89c7b2-b9d2-4105-be8a-07232645205d-kube-api-access-x8fx6\") pod \"payload-processing-e2e-api-b-66f145-b99dc85-bzg66\" (UID: \"0a89c7b2-b9d2-4105-be8a-07232645205d\") " pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:43:14.619420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.619397    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"]
Nov 03 16:43:14.687449 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.687396    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-66f145\""
Nov 03 16:43:14.687657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.687543    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-66f145-dockercfg-w8zf2\""
Nov 03 16:43:14.719943 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.719909    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8fx6\" (UniqueName: \"kubernetes.io/projected/0a89c7b2-b9d2-4105-be8a-07232645205d-kube-api-access-x8fx6\") pod \"payload-processing-e2e-api-b-66f145-b99dc85-bzg66\" (UID: \"0a89c7b2-b9d2-4105-be8a-07232645205d\") " pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:43:14.720236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.720214    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a89c7b2-b9d2-4105-be8a-07232645205d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-66f145-b99dc85-bzg66\" (UID: \"0a89c7b2-b9d2-4105-be8a-07232645205d\") " pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:43:14.721296 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.721251    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a89c7b2-b9d2-4105-be8a-07232645205d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-66f145-b99dc85-bzg66\" (UID: \"0a89c7b2-b9d2-4105-be8a-07232645205d\") " pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:43:14.782469 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.782441    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8fx6\" (UniqueName: \"kubernetes.io/projected/0a89c7b2-b9d2-4105-be8a-07232645205d-kube-api-access-x8fx6\") pod \"payload-processing-e2e-api-b-66f145-b99dc85-bzg66\" (UID: \"0a89c7b2-b9d2-4105-be8a-07232645205d\") " pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:43:14.922329 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:14.922296    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:43:15.308245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:15.308219    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"]
Nov 03 16:43:15.310841 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:43:15.310811    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a89c7b2_b9d2_4105_be8a_07232645205d.slice/crio-1216156af15f96f93b024670b48636253e7d13a880b8d263c2204c0dc6fbe268 WatchSource:0}: Error finding container 1216156af15f96f93b024670b48636253e7d13a880b8d263c2204c0dc6fbe268: Status 404 returned error can't find the container with id 1216156af15f96f93b024670b48636253e7d13a880b8d263c2204c0dc6fbe268
Nov 03 16:43:15.457038 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:15.457001    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66" event={"ID":"0a89c7b2-b9d2-4105-be8a-07232645205d","Type":"ContainerStarted","Data":"d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1"}
Nov 03 16:43:15.457222 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:15.457073    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:43:15.457222 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:15.457099    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66" event={"ID":"0a89c7b2-b9d2-4105-be8a-07232645205d","Type":"ContainerStarted","Data":"1216156af15f96f93b024670b48636253e7d13a880b8d263c2204c0dc6fbe268"}
Nov 03 16:43:15.580254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:15.580158    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66" podStartSLOduration=1.580144687 podStartE2EDuration="1.580144687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:43:15.576487727 +0000 UTC m=+1881.400938995" watchObservedRunningTime="2026-11-03 16:43:15.580144687 +0000 UTC m=+1881.404595992"
Nov 03 16:43:26.467622 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:43:26.467593    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:43:33.395974 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:43:33.395925    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:44:00.197109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.197023    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"]
Nov 03 16:44:00.820889 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.820852    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"]
Nov 03 16:44:00.826114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.826095    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:00.903134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.903100    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6f58cf-openshift-default-dockercfg-8hhcg\""
Nov 03 16:44:00.932797 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.932755    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-podinfo\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:00.934181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.934154    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-certs\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:00.934567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.934536    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-envoy\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:00.934823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.934805    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt25k\" (UniqueName: \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-kube-api-access-jt25k\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:00.943211 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.943183    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-credential-socket\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:00.943495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.943475    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-data\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:00.943660 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.943645    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-socket\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:00.943856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.943838    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istiod-ca-cert\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:00.944017 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.944004    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-token\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:00.959020 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:00.958373    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"]
Nov 03 16:44:01.073804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.046981    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-podinfo\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.085944 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.085903    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-certs\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.093107 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.093056    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-envoy\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.093410 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.093388    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt25k\" (UniqueName: \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-kube-api-access-jt25k\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.093810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.093788    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-credential-socket\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.093976 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.093958    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-data\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.094104 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.094086    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-socket\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.094258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.094243    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istiod-ca-cert\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.094386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.094371    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-token\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.101218 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.101193    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-credential-socket\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.101938 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.101917    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-certs\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.102762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.102334    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-data\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.102890 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.102537    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-socket\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.108528 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.108499    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istiod-ca-cert\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.110861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.110836    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-podinfo\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.140026 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.139100    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-envoy\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.182875 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.182257    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt25k\" (UniqueName: \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-kube-api-access-jt25k\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.182875 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.182583    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-token\") pod \"e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") " pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.248487 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.248458    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:01.694208 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.694175    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"]
Nov 03 16:44:01.695036 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:44:01.695010    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8114cb0a_6d4e_4b87_b562_03cf2fec6319.slice/crio-a4678da25a569411300f2d05dec243611ce01ab1740a643edd69defd2172dab3 WatchSource:0}: Error finding container a4678da25a569411300f2d05dec243611ce01ab1740a643edd69defd2172dab3: Status 404 returned error can't find the container with id a4678da25a569411300f2d05dec243611ce01ab1740a643edd69defd2172dab3
Nov 03 16:44:01.697820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.697785    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:44:01.697901 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.697852    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:44:01.697901 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.697885    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:44:01.980811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.974096    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" event={"ID":"8114cb0a-6d4e-4b87-b562-03cf2fec6319","Type":"ContainerStarted","Data":"73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531"}
Nov 03 16:44:01.980811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:01.974164    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" event={"ID":"8114cb0a-6d4e-4b87-b562-03cf2fec6319","Type":"ContainerStarted","Data":"a4678da25a569411300f2d05dec243611ce01ab1740a643edd69defd2172dab3"}
Nov 03 16:44:02.135407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:02.133611    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" podStartSLOduration=2.133588493 podStartE2EDuration="2.133588493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:44:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:44:02.10628674 +0000 UTC m=+1927.930738011" watchObservedRunningTime="2026-11-03 16:44:02.133588493 +0000 UTC m=+1927.958039766"
Nov 03 16:44:02.248941 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:02.248793    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:02.251903 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:02.251874    5605 patch_prober.go:29] interesting pod/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Nov 03 16:44:02.252154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:02.252123    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Nov 03 16:44:03.254201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:03.254169    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:03.424242 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:44:03.424206    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:44:03.988079 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:03.988047    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:03.988967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:03.988948    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:44:05.198224 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:05.198183    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg" podUID="7ce2633c-8018-433b-8afc-7928e26a560b" containerName="payload-processing" containerID="cri-o://96dc0b5f0d8524e969a4746543c4a68b77db0555e1a1952a204cedcd14163c7c" gracePeriod=25
Nov 03 16:44:05.489136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:05.489050    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"]
Nov 03 16:44:05.489309 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:05.489265    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy" containerID="cri-o://f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385" gracePeriod=30
Nov 03 16:44:07.983110 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:07.983067    5605 patch_prober.go:29] interesting pod/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:44:07.983611 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:07.983147    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:44:10.773455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.773384    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:44:10.800606 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.800570    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-credential-socket\") pod \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") "
Nov 03 16:44:10.800766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.800631    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-kube-api-access-mlkrx\" (UniqueName: \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-kube-api-access-mlkrx\") pod \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") "
Nov 03 16:44:10.800766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.800692    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-data\") pod \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") "
Nov 03 16:44:10.800903 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.800792    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-socket\") pod \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") "
Nov 03 16:44:10.800903 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.800822    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-token\" (UniqueName: \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-token\") pod \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") "
Nov 03 16:44:10.800903 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.800863    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-credential-socket" pod "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" (UID: "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:44:10.801053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.800939    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-podinfo\") pod \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") "
Nov 03 16:44:10.801111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.801047    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-data" pod "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" (UID: "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:44:10.801111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.801087    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-socket" pod "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" (UID: "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:44:10.801218 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.801081    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istiod-ca-cert\") pod \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") "
Nov 03 16:44:10.801218 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.801161    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-certs\") pod \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") "
Nov 03 16:44:10.801322 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.801276    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-envoy\") pod \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\" (UID: \"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d\") "
Nov 03 16:44:10.801383 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.801355    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-certs" pod "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" (UID: "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:44:10.801439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.801414    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istiod-ca-cert" pod "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" (UID: "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:44:10.802254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.802224    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:10.802382 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.802258    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:10.802382 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.802273    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:10.802382 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.802288    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:10.802382 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.802302    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:10.803208 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.803176    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-kube-api-access-mlkrx" pod "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" (UID: "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d"). InnerVolumeSpecName "kube-api-access-mlkrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:44:10.803337 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.803219    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-token" pod "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" (UID: "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:44:10.803383 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.803339    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-podinfo" pod "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" (UID: "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:44:10.803561 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.803541    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-envoy" pod "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" (UID: "5c2fae7c-b4c0-49cf-82a0-c93f475ae57d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:44:10.903756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.903702    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlkrx\" (UniqueName: \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-kube-api-access-mlkrx\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:10.903756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.903757    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:10.903980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.903771    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:10.903980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:10.903784    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:11.081705 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:11.081615    5605 generic.go:356] "Generic (PLEG): container finished" podID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerID="f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385" exitCode=0
Nov 03 16:44:11.081705 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:11.081665    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"
Nov 03 16:44:11.081705 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:11.081691    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" event={"ID":"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d","Type":"ContainerDied","Data":"f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385"}
Nov 03 16:44:11.082018 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:11.081809    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw" event={"ID":"5c2fae7c-b4c0-49cf-82a0-c93f475ae57d","Type":"ContainerDied","Data":"dbbe61c223c23a8dd6512f5614df1262e4df5992b862034ecd6740897a2dc817"}
Nov 03 16:44:11.082018 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:11.081846    5605 scope.go:122] "RemoveContainer" containerID="f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385"
Nov 03 16:44:11.096529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:11.095587    5605 scope.go:122] "RemoveContainer" containerID="f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385"
Nov 03 16:44:11.096529 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:44:11.096483    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385\": container with ID starting with f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385 not found: ID does not exist" containerID="f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385"
Nov 03 16:44:11.096529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:11.096509    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385"} err="failed to get container status \"f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385\": rpc error: code = NotFound desc = could not find container \"f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385\": container with ID starting with f70f1f0aee2ea443af7f943c6b561811d726e62af9457f7f85a10eda1b382385 not found: ID does not exist"
Nov 03 16:44:11.259046 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:11.258996    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"]
Nov 03 16:44:11.263551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:11.263530    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-dcchw"]
Nov 03 16:44:11.770084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:11.770051    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg" podUID="7ce2633c-8018-433b-8afc-7928e26a560b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 16:44:12.309893 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:12.309861    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" path="/var/lib/kubelet/pods/5c2fae7c-b4c0-49cf-82a0-c93f475ae57d/volumes"
Nov 03 16:44:13.773753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.769427    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"]
Nov 03 16:44:13.774496 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.774468    5605 cpu_manager.go:414] "containerMap: removing container" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy"
Nov 03 16:44:13.774901 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.774880    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy"
Nov 03 16:44:13.775457 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.775442    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c2fae7c-b4c0-49cf-82a0-c93f475ae57d" containerName="istio-proxy"
Nov 03 16:44:13.780205 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.780182    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:44:13.849978 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.849935    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"]
Nov 03 16:44:13.895804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.888557    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4snbg\" (UniqueName: \"kubernetes.io/projected/36f78ba2-86af-4225-ba2e-35f3d772e34a-kube-api-access-4snbg\") pod \"payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf\" (UID: \"36f78ba2-86af-4225-ba2e-35f3d772e34a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:44:13.896337 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.896297    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36f78ba2-86af-4225-ba2e-35f3d772e34a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf\" (UID: \"36f78ba2-86af-4225-ba2e-35f3d772e34a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:44:13.897025 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.897001    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-6f58cf-dockercfg-h7tqh\""
Nov 03 16:44:13.897242 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.897225    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-6f58cf\""
Nov 03 16:44:13.929923 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:13.912347    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"]
Nov 03 16:44:14.006800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.006761    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36f78ba2-86af-4225-ba2e-35f3d772e34a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf\" (UID: \"36f78ba2-86af-4225-ba2e-35f3d772e34a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:44:14.007314 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.007287    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4snbg\" (UniqueName: \"kubernetes.io/projected/36f78ba2-86af-4225-ba2e-35f3d772e34a-kube-api-access-4snbg\") pod \"payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf\" (UID: \"36f78ba2-86af-4225-ba2e-35f3d772e34a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:44:14.025810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.008878    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36f78ba2-86af-4225-ba2e-35f3d772e34a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf\" (UID: \"36f78ba2-86af-4225-ba2e-35f3d772e34a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:44:14.028754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.028628    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:44:14.063806 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.062656    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"]
Nov 03 16:44:14.120073 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.119953    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4snbg\" (UniqueName: \"kubernetes.io/projected/36f78ba2-86af-4225-ba2e-35f3d772e34a-kube-api-access-4snbg\") pod \"payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf\" (UID: \"36f78ba2-86af-4225-ba2e-35f3d772e34a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:44:14.120555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.120350    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtht9\" (UniqueName: \"kubernetes.io/projected/89e47de6-1d71-460d-8323-34aa5222eb6b-kube-api-access-xtht9\") pod \"payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x\" (UID: \"89e47de6-1d71-460d-8323-34aa5222eb6b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:44:14.120555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.120469    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89e47de6-1d71-460d-8323-34aa5222eb6b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x\" (UID: \"89e47de6-1d71-460d-8323-34aa5222eb6b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:44:14.219753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.213922    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:44:14.241770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.241093    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89e47de6-1d71-460d-8323-34aa5222eb6b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x\" (UID: \"89e47de6-1d71-460d-8323-34aa5222eb6b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:44:14.241770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.241397    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtht9\" (UniqueName: \"kubernetes.io/projected/89e47de6-1d71-460d-8323-34aa5222eb6b-kube-api-access-xtht9\") pod \"payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x\" (UID: \"89e47de6-1d71-460d-8323-34aa5222eb6b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:44:14.253067 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.242670    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89e47de6-1d71-460d-8323-34aa5222eb6b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x\" (UID: \"89e47de6-1d71-460d-8323-34aa5222eb6b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:44:14.333254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.333110    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtht9\" (UniqueName: \"kubernetes.io/projected/89e47de6-1d71-460d-8323-34aa5222eb6b-kube-api-access-xtht9\") pod \"payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x\" (UID: \"89e47de6-1d71-460d-8323-34aa5222eb6b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:44:14.337638 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.337605    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:44:14.749024 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:44:14.748985    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f78ba2_86af_4225_ba2e_35f3d772e34a.slice/crio-ce836f1838b6d8bb05dd8cbf2204ce016bbb4469a94d48df3cd07af00e627115 WatchSource:0}: Error finding container ce836f1838b6d8bb05dd8cbf2204ce016bbb4469a94d48df3cd07af00e627115: Status 404 returned error can't find the container with id ce836f1838b6d8bb05dd8cbf2204ce016bbb4469a94d48df3cd07af00e627115
Nov 03 16:44:14.749439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.749414    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"]
Nov 03 16:44:14.810384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:14.810351    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"]
Nov 03 16:44:14.813897 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:44:14.812899    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e47de6_1d71_460d_8323_34aa5222eb6b.slice/crio-cd57343528ad5d46f76f00eb750981bfce245e9ab7746059946dea4dbb41c3d0 WatchSource:0}: Error finding container cd57343528ad5d46f76f00eb750981bfce245e9ab7746059946dea4dbb41c3d0: Status 404 returned error can't find the container with id cd57343528ad5d46f76f00eb750981bfce245e9ab7746059946dea4dbb41c3d0
Nov 03 16:44:15.272870 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:15.272829    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x" event={"ID":"89e47de6-1d71-460d-8323-34aa5222eb6b","Type":"ContainerStarted","Data":"40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b"}
Nov 03 16:44:15.273053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:15.272897    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x" event={"ID":"89e47de6-1d71-460d-8323-34aa5222eb6b","Type":"ContainerStarted","Data":"cd57343528ad5d46f76f00eb750981bfce245e9ab7746059946dea4dbb41c3d0"}
Nov 03 16:44:15.273053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:15.272930    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:44:15.274669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:15.274642    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf" event={"ID":"36f78ba2-86af-4225-ba2e-35f3d772e34a","Type":"ContainerStarted","Data":"40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee"}
Nov 03 16:44:15.274789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:15.274689    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf" event={"ID":"36f78ba2-86af-4225-ba2e-35f3d772e34a","Type":"ContainerStarted","Data":"ce836f1838b6d8bb05dd8cbf2204ce016bbb4469a94d48df3cd07af00e627115"}
Nov 03 16:44:15.274789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:15.274715    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:44:15.539998 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:15.539682    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf" podStartSLOduration=2.539662506 podStartE2EDuration="2.539662506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:44:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:44:15.537259575 +0000 UTC m=+1941.361710847" watchObservedRunningTime="2026-11-03 16:44:15.539662506 +0000 UTC m=+1941.364113778"
Nov 03 16:44:15.547768 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:15.547649    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x" podStartSLOduration=2.5476325859999998 podStartE2EDuration="2.547632586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:44:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:44:15.406469054 +0000 UTC m=+1941.230920326" watchObservedRunningTime="2026-11-03 16:44:15.547632586 +0000 UTC m=+1941.372083858"
Nov 03 16:44:21.770879 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:21.770840    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg" podUID="7ce2633c-8018-433b-8afc-7928e26a560b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 16:44:21.857369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:21.857336    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"]
Nov 03 16:44:21.958760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:21.958709    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"]
Nov 03 16:44:23.254206 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:23.252774    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": read tcp 10.133.0.2:45760->10.133.0.114:15021: read: connection reset by peer" start-of-body=
Nov 03 16:44:23.254206 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:23.252843    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": read tcp 10.133.0.2:45760->10.133.0.114:15021: read: connection reset by peer"
Nov 03 16:44:23.477642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:23.477558    5605 generic.go:356] "Generic (PLEG): container finished" podID="71409250-bb69-4586-8dc9-d22c888f1160" containerID="eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b" exitCode=0
Nov 03 16:44:23.478355 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:23.478325    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" event={"ID":"71409250-bb69-4586-8dc9-d22c888f1160","Type":"ContainerDied","Data":"eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b"}
Nov 03 16:44:23.479079 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:23.479042    5605 scope.go:122] "RemoveContainer" containerID="eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b"
Nov 03 16:44:23.482704 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:23.482660    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:44:23.495298 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:23.495263    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:44:23.495502 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:23.495482    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:44:24.489410 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:24.489376    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" event={"ID":"71409250-bb69-4586-8dc9-d22c888f1160","Type":"ContainerStarted","Data":"af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3"}
Nov 03 16:44:24.687226 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:24.687192    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:44:24.687594 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:24.687566    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Nov 03 16:44:24.687794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:24.687768    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Nov 03 16:44:25.512694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.512660    5605 generic.go:356] "Generic (PLEG): container finished" podID="7ce2633c-8018-433b-8afc-7928e26a560b" containerID="96dc0b5f0d8524e969a4746543c4a68b77db0555e1a1952a204cedcd14163c7c" exitCode=0
Nov 03 16:44:25.513112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.512757    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg" event={"ID":"7ce2633c-8018-433b-8afc-7928e26a560b","Type":"ContainerDied","Data":"96dc0b5f0d8524e969a4746543c4a68b77db0555e1a1952a204cedcd14163c7c"}
Nov 03 16:44:25.513112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.512825    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg" event={"ID":"7ce2633c-8018-433b-8afc-7928e26a560b","Type":"ContainerDied","Data":"c0c1a6ab4ca7cf4d1b500b5644b42d0ab7f5d69b7656f5eba385e176e888fa33"}
Nov 03 16:44:25.513112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.512852    5605 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0c1a6ab4ca7cf4d1b500b5644b42d0ab7f5d69b7656f5eba385e176e888fa33"
Nov 03 16:44:25.530178 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.530153    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:44:25.619919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.619889    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ce2633c-8018-433b-8afc-7928e26a560b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce2633c-8018-433b-8afc-7928e26a560b-plugins-config-volume\") pod \"7ce2633c-8018-433b-8afc-7928e26a560b\" (UID: \"7ce2633c-8018-433b-8afc-7928e26a560b\") "
Nov 03 16:44:25.620098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.620000    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ce2633c-8018-433b-8afc-7928e26a560b-kube-api-access-rpns6\" (UniqueName: \"kubernetes.io/projected/7ce2633c-8018-433b-8afc-7928e26a560b-kube-api-access-rpns6\") pod \"7ce2633c-8018-433b-8afc-7928e26a560b\" (UID: \"7ce2633c-8018-433b-8afc-7928e26a560b\") "
Nov 03 16:44:25.620290 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.620263    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce2633c-8018-433b-8afc-7928e26a560b-plugins-config-volume" pod "7ce2633c-8018-433b-8afc-7928e26a560b" (UID: "7ce2633c-8018-433b-8afc-7928e26a560b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:44:25.620821 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.620799    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce2633c-8018-433b-8afc-7928e26a560b-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:25.622518 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.622495    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce2633c-8018-433b-8afc-7928e26a560b-kube-api-access-rpns6" pod "7ce2633c-8018-433b-8afc-7928e26a560b" (UID: "7ce2633c-8018-433b-8afc-7928e26a560b"). InnerVolumeSpecName "kube-api-access-rpns6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:44:25.702660 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.702186    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Nov 03 16:44:25.702660 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.702260    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Nov 03 16:44:25.727868 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:25.727535    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpns6\" (UniqueName: \"kubernetes.io/projected/7ce2633c-8018-433b-8afc-7928e26a560b-kube-api-access-rpns6\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:26.318303 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:26.318264    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:44:26.318489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:26.318346    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:44:26.523844 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:26.523813    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"
Nov 03 16:44:26.715900 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:26.715850    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Nov 03 16:44:26.716133 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:26.715940    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Nov 03 16:44:26.718023 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:26.717993    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"]
Nov 03 16:44:26.721398 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:26.721376    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9635db0d-8997d67c-rq9kg"]
Nov 03 16:44:26.858465 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:26.858421    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b" podUID="205bb884-1493-4036-9e7d-5a995c5826cb" containerName="payload-processing" containerID="cri-o://4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8" gracePeriod=25
Nov 03 16:44:26.959431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:26.959397    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm" podUID="3fcf4a33-5dcb-4573-9581-ad772f3d2ebc" containerName="payload-pre-processing" containerID="cri-o://4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900" gracePeriod=25
Nov 03 16:44:27.221153 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.221127    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:44:27.350262 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.350232    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/205bb884-1493-4036-9e7d-5a995c5826cb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205bb884-1493-4036-9e7d-5a995c5826cb-plugins-config-volume\") pod \"205bb884-1493-4036-9e7d-5a995c5826cb\" (UID: \"205bb884-1493-4036-9e7d-5a995c5826cb\") "
Nov 03 16:44:27.350432 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.350397    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/205bb884-1493-4036-9e7d-5a995c5826cb-kube-api-access-tj4lf\" (UniqueName: \"kubernetes.io/projected/205bb884-1493-4036-9e7d-5a995c5826cb-kube-api-access-tj4lf\") pod \"205bb884-1493-4036-9e7d-5a995c5826cb\" (UID: \"205bb884-1493-4036-9e7d-5a995c5826cb\") "
Nov 03 16:44:27.350925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.350892    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/205bb884-1493-4036-9e7d-5a995c5826cb-plugins-config-volume" pod "205bb884-1493-4036-9e7d-5a995c5826cb" (UID: "205bb884-1493-4036-9e7d-5a995c5826cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:44:27.357799 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.353358    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205bb884-1493-4036-9e7d-5a995c5826cb-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:27.357799 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.353892    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/205bb884-1493-4036-9e7d-5a995c5826cb-kube-api-access-tj4lf" pod "205bb884-1493-4036-9e7d-5a995c5826cb" (UID: "205bb884-1493-4036-9e7d-5a995c5826cb"). InnerVolumeSpecName "kube-api-access-tj4lf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:44:27.357799 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.356020    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:44:27.454167 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.454131    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-plugins-config-volume\") pod \"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc\" (UID: \"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc\") "
Nov 03 16:44:27.454363 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.454274    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-kube-api-access-qx469\" (UniqueName: \"kubernetes.io/projected/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-kube-api-access-qx469\") pod \"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc\" (UID: \"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc\") "
Nov 03 16:44:27.454593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.454567    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-plugins-config-volume" pod "3fcf4a33-5dcb-4573-9581-ad772f3d2ebc" (UID: "3fcf4a33-5dcb-4573-9581-ad772f3d2ebc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:44:27.455296 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.455262    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:27.455296 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.455297    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj4lf\" (UniqueName: \"kubernetes.io/projected/205bb884-1493-4036-9e7d-5a995c5826cb-kube-api-access-tj4lf\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:27.456714 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.456689    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-kube-api-access-qx469" pod "3fcf4a33-5dcb-4573-9581-ad772f3d2ebc" (UID: "3fcf4a33-5dcb-4573-9581-ad772f3d2ebc"). InnerVolumeSpecName "kube-api-access-qx469". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:44:27.534310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.534210    5605 generic.go:356] "Generic (PLEG): container finished" podID="3fcf4a33-5dcb-4573-9581-ad772f3d2ebc" containerID="4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900" exitCode=0
Nov 03 16:44:27.534310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.534273    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"
Nov 03 16:44:27.534310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.534291    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm" event={"ID":"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc","Type":"ContainerDied","Data":"4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900"}
Nov 03 16:44:27.534925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.534375    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm" event={"ID":"3fcf4a33-5dcb-4573-9581-ad772f3d2ebc","Type":"ContainerDied","Data":"5a81f43eb4abe48039a6008774bfd7962ecf051c03315c566aec90a9a7f98917"}
Nov 03 16:44:27.534925 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.534408    5605 scope.go:122] "RemoveContainer" containerID="4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900"
Nov 03 16:44:27.535979 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.535957    5605 generic.go:356] "Generic (PLEG): container finished" podID="205bb884-1493-4036-9e7d-5a995c5826cb" containerID="4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8" exitCode=0
Nov 03 16:44:27.536079 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.536019    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b" event={"ID":"205bb884-1493-4036-9e7d-5a995c5826cb","Type":"ContainerDied","Data":"4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8"}
Nov 03 16:44:27.536079 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.536075    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b" event={"ID":"205bb884-1493-4036-9e7d-5a995c5826cb","Type":"ContainerDied","Data":"d16d40fe0bf5c84fb6afbcceefa4b15f9795a09066b8fcae78a77edbef3a0e68"}
Nov 03 16:44:27.536215 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.536089    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"
Nov 03 16:44:27.549010 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.548887    5605 scope.go:122] "RemoveContainer" containerID="4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900"
Nov 03 16:44:27.549195 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:44:27.549176    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900\": container with ID starting with 4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900 not found: ID does not exist" containerID="4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900"
Nov 03 16:44:27.549261 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.549204    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900"} err="failed to get container status \"4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900\": rpc error: code = NotFound desc = could not find container \"4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900\": container with ID starting with 4c8858c3f2a3c095869494bb8aee60344c66e00c445a5809c6b910bafd3a1900 not found: ID does not exist"
Nov 03 16:44:27.549261 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.549222    5605 scope.go:122] "RemoveContainer" containerID="4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8"
Nov 03 16:44:27.556924 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.556892    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx469\" (UniqueName: \"kubernetes.io/projected/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc-kube-api-access-qx469\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:27.649945 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.648278    5605 scope.go:122] "RemoveContainer" containerID="4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8"
Nov 03 16:44:27.649945 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:44:27.649530    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8\": container with ID starting with 4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8 not found: ID does not exist" containerID="4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8"
Nov 03 16:44:27.649945 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.649566    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8"} err="failed to get container status \"4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8\": rpc error: code = NotFound desc = could not find container \"4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8\": container with ID starting with 4dee43e1067518798c16eebc0c6d91abaa0428f92b6e53cd0c1c4e802b357ff8 not found: ID does not exist"
Nov 03 16:44:27.709789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.699183    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Nov 03 16:44:27.709789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.699256    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Nov 03 16:44:27.755844 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.755802    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"]
Nov 03 16:44:27.757655 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.757619    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-438248-67bc76f475-25vdm"]
Nov 03 16:44:27.854596 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.854562    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"]
Nov 03 16:44:27.862879 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:27.862842    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-438248-6f678dfc-dbv2b"]
Nov 03 16:44:28.320425 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:28.320354    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="205bb884-1493-4036-9e7d-5a995c5826cb" path="/var/lib/kubelet/pods/205bb884-1493-4036-9e7d-5a995c5826cb/volumes"
Nov 03 16:44:28.320864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:28.320850    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3fcf4a33-5dcb-4573-9581-ad772f3d2ebc" path="/var/lib/kubelet/pods/3fcf4a33-5dcb-4573-9581-ad772f3d2ebc/volumes"
Nov 03 16:44:28.321246 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:28.321231    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ce2633c-8018-433b-8afc-7928e26a560b" path="/var/lib/kubelet/pods/7ce2633c-8018-433b-8afc-7928e26a560b/volumes"
Nov 03 16:44:28.691131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:28.691093    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Nov 03 16:44:28.691701 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:28.691670    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Nov 03 16:44:29.701753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:29.701188    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused" start-of-body=
Nov 03 16:44:29.701753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:29.701254    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.114:15021/healthz/ready\": dial tcp 10.133.0.114:15021: connect: connection refused"
Nov 03 16:44:30.691837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:30.691804    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:44:30.692136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:30.692113    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:44:30.692807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:30.692785    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:44:33.395495 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:44:33.395461    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:44:51.762240 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:51.762198    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"]
Nov 03 16:44:51.762919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:51.762458    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" podUID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" containerName="istio-proxy" containerID="cri-o://ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a" gracePeriod=30
Nov 03 16:44:55.240632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:55.240601    5605 scope.go:122] "RemoveContainer" containerID="96dc0b5f0d8524e969a4746543c4a68b77db0555e1a1952a204cedcd14163c7c"
Nov 03 16:44:57.051224 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.051195    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:44:57.161809 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.161767    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-kube-api-access-vjnmx\" (UniqueName: \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-kube-api-access-vjnmx\") pod \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") "
Nov 03 16:44:57.162101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.161885    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istiod-ca-cert\") pod \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") "
Nov 03 16:44:57.162101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.161941    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-credential-socket\") pod \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") "
Nov 03 16:44:57.162101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.161969    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-data\") pod \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") "
Nov 03 16:44:57.162101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.162023    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-envoy\") pod \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") "
Nov 03 16:44:57.162101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.162077    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-certs\") pod \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") "
Nov 03 16:44:57.162414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.162116    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-podinfo\") pod \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") "
Nov 03 16:44:57.162414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.162172    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-token\" (UniqueName: \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-token\") pod \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") "
Nov 03 16:44:57.162414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.162215    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-socket\") pod \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\" (UID: \"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7\") "
Nov 03 16:44:57.162414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.162253    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-credential-socket" pod "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" (UID: "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:44:57.162414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.162359    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-data" pod "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" (UID: "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:44:57.162414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.162393    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istiod-ca-cert" pod "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" (UID: "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:44:57.162414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.162400    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-certs" pod "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" (UID: "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:44:57.162826 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.162681    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-socket" pod "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" (UID: "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:44:57.163277 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.163254    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:57.163368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.163283    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:57.163368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.163298    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:57.163368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.163312    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:57.163368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.163327    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:57.164669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.164639    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-kube-api-access-vjnmx" pod "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" (UID: "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7"). InnerVolumeSpecName "kube-api-access-vjnmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:44:57.164848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.164768    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-envoy" pod "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" (UID: "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:44:57.165139 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.165119    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-podinfo" pod "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" (UID: "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:44:57.165213 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.165148    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-token" pod "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" (UID: "5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:44:57.265529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.265490    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:57.265529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.265532    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:57.265761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.265548    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:57.265761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.265563    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjnmx\" (UniqueName: \"kubernetes.io/projected/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7-kube-api-access-vjnmx\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:44:57.784263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.784223    5605 generic.go:356] "Generic (PLEG): container finished" podID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" containerID="ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a" exitCode=0
Nov 03 16:44:57.784437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.784266    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"
Nov 03 16:44:57.784437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.784308    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" event={"ID":"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7","Type":"ContainerDied","Data":"ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a"}
Nov 03 16:44:57.784437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.784360    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q" event={"ID":"5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7","Type":"ContainerDied","Data":"2aebb24a89fe49b80d0beeb7c17e0ac1501b54b02ba45ffc31988098ae291d2c"}
Nov 03 16:44:57.784437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.784383    5605 scope.go:122] "RemoveContainer" containerID="ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a"
Nov 03 16:44:57.796083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.796066    5605 scope.go:122] "RemoveContainer" containerID="ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a"
Nov 03 16:44:57.796369 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:44:57.796344    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a\": container with ID starting with ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a not found: ID does not exist" containerID="ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a"
Nov 03 16:44:57.796446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.796377    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a"} err="failed to get container status \"ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a\": rpc error: code = NotFound desc = could not find container \"ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a\": container with ID starting with ea51b93f0edac17a8d7a0ff1701f0b727809af3e108645831f1ba9b46defe14a not found: ID does not exist"
Nov 03 16:44:57.957807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:57.957770    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"]
Nov 03 16:44:58.018693 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:58.018646    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-zp95q"]
Nov 03 16:44:58.310644 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:44:58.310611    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" path="/var/lib/kubelet/pods/5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7/volumes"
Nov 03 16:45:00.271074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.271033    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"]
Nov 03 16:45:00.271954 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.271930    5605 cpu_manager.go:414] "containerMap: removing container" podUID="7ce2633c-8018-433b-8afc-7928e26a560b" containerName="payload-processing"
Nov 03 16:45:00.271954 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.271953    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ce2633c-8018-433b-8afc-7928e26a560b" containerName="payload-processing"
Nov 03 16:45:00.272135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.271967    5605 cpu_manager.go:414] "containerMap: removing container" podUID="205bb884-1493-4036-9e7d-5a995c5826cb" containerName="payload-processing"
Nov 03 16:45:00.272135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.271978    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="205bb884-1493-4036-9e7d-5a995c5826cb" containerName="payload-processing"
Nov 03 16:45:00.272135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.272037    5605 cpu_manager.go:414] "containerMap: removing container" podUID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" containerName="istio-proxy"
Nov 03 16:45:00.272135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.272045    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" containerName="istio-proxy"
Nov 03 16:45:00.272135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.272079    5605 cpu_manager.go:414] "containerMap: removing container" podUID="3fcf4a33-5dcb-4573-9581-ad772f3d2ebc" containerName="payload-pre-processing"
Nov 03 16:45:00.272135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.272087    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fcf4a33-5dcb-4573-9581-ad772f3d2ebc" containerName="payload-pre-processing"
Nov 03 16:45:00.272420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.272263    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ce2633c-8018-433b-8afc-7928e26a560b" containerName="payload-processing"
Nov 03 16:45:00.272420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.272276    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="205bb884-1493-4036-9e7d-5a995c5826cb" containerName="payload-processing"
Nov 03 16:45:00.272420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.272294    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="5666ee98-cdcd-4ce9-8eb1-cfa5f5d9c6c7" containerName="istio-proxy"
Nov 03 16:45:00.272420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.272309    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fcf4a33-5dcb-4573-9581-ad772f3d2ebc" containerName="payload-pre-processing"
Nov 03 16:45:00.275395 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.275371    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"
Nov 03 16:45:00.287963 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.287936    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"]
Nov 03 16:45:00.292050 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.292029    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"
Nov 03 16:45:00.342426 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.342390    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fnmxp\""
Nov 03 16:45:00.362831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.362792    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"]
Nov 03 16:45:00.367310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.367279    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"]
Nov 03 16:45:00.414311 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.414279    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b70f65a-9f12-458d-b942-61b7e178f019-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c\" (UID: \"0b70f65a-9f12-458d-b942-61b7e178f019\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"
Nov 03 16:45:00.414495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.414331    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4wrg\" (UniqueName: \"kubernetes.io/projected/0b70f65a-9f12-458d-b942-61b7e178f019-kube-api-access-l4wrg\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c\" (UID: \"0b70f65a-9f12-458d-b942-61b7e178f019\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"
Nov 03 16:45:00.414495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.414477    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c8dv\" (UniqueName: \"kubernetes.io/projected/6747825c-9319-4280-8201-62063c3aa4f8-kube-api-access-8c8dv\") pod \"maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7\" (UID: \"6747825c-9319-4280-8201-62063c3aa4f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"
Nov 03 16:45:00.414589 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.414569    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6747825c-9319-4280-8201-62063c3aa4f8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7\" (UID: \"6747825c-9319-4280-8201-62063c3aa4f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"
Nov 03 16:45:00.516172 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.516138    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b70f65a-9f12-458d-b942-61b7e178f019-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c\" (UID: \"0b70f65a-9f12-458d-b942-61b7e178f019\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"
Nov 03 16:45:00.516359 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.516181    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4wrg\" (UniqueName: \"kubernetes.io/projected/0b70f65a-9f12-458d-b942-61b7e178f019-kube-api-access-l4wrg\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c\" (UID: \"0b70f65a-9f12-458d-b942-61b7e178f019\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"
Nov 03 16:45:00.516359 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.516308    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c8dv\" (UniqueName: \"kubernetes.io/projected/6747825c-9319-4280-8201-62063c3aa4f8-kube-api-access-8c8dv\") pod \"maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7\" (UID: \"6747825c-9319-4280-8201-62063c3aa4f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"
Nov 03 16:45:00.516496 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.516379    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6747825c-9319-4280-8201-62063c3aa4f8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7\" (UID: \"6747825c-9319-4280-8201-62063c3aa4f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"
Nov 03 16:45:00.516881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.516851    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b70f65a-9f12-458d-b942-61b7e178f019-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c\" (UID: \"0b70f65a-9f12-458d-b942-61b7e178f019\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"
Nov 03 16:45:00.517075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.517053    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6747825c-9319-4280-8201-62063c3aa4f8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7\" (UID: \"6747825c-9319-4280-8201-62063c3aa4f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"
Nov 03 16:45:00.578308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.578006    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4wrg\" (UniqueName: \"kubernetes.io/projected/0b70f65a-9f12-458d-b942-61b7e178f019-kube-api-access-l4wrg\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c\" (UID: \"0b70f65a-9f12-458d-b942-61b7e178f019\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"
Nov 03 16:45:00.578308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.578254    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c8dv\" (UniqueName: \"kubernetes.io/projected/6747825c-9319-4280-8201-62063c3aa4f8-kube-api-access-8c8dv\") pod \"maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7\" (UID: \"6747825c-9319-4280-8201-62063c3aa4f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"
Nov 03 16:45:00.590934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.590911    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"
Nov 03 16:45:00.607693 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.607663    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"
Nov 03 16:45:00.976802 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:00.976774    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"]
Nov 03 16:45:00.978042 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:45:00.978014    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6747825c_9319_4280_8201_62063c3aa4f8.slice/crio-abc105dc2ea5111e42a20de5ace98e394fdf0e7381df763b29ee6c917a6a5a52 WatchSource:0}: Error finding container abc105dc2ea5111e42a20de5ace98e394fdf0e7381df763b29ee6c917a6a5a52: Status 404 returned error can't find the container with id abc105dc2ea5111e42a20de5ace98e394fdf0e7381df763b29ee6c917a6a5a52
Nov 03 16:45:01.028165 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:01.028136    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"]
Nov 03 16:45:01.029172 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:45:01.029149    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b70f65a_9f12_458d_b942_61b7e178f019.slice/crio-01cd62386cda5f3804f194e5905699ad6e497448c54adf9d9cf4857049774f04 WatchSource:0}: Error finding container 01cd62386cda5f3804f194e5905699ad6e497448c54adf9d9cf4857049774f04: Status 404 returned error can't find the container with id 01cd62386cda5f3804f194e5905699ad6e497448c54adf9d9cf4857049774f04
Nov 03 16:45:01.817744 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:01.817691    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" event={"ID":"6747825c-9319-4280-8201-62063c3aa4f8","Type":"ContainerStarted","Data":"bfda005fb0a401fd84088204ada73336155403ddbff067dbff92ed29ec4843b2"}
Nov 03 16:45:01.818213 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:01.817802    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" event={"ID":"6747825c-9319-4280-8201-62063c3aa4f8","Type":"ContainerStarted","Data":"abc105dc2ea5111e42a20de5ace98e394fdf0e7381df763b29ee6c917a6a5a52"}
Nov 03 16:45:01.819417 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:01.819389    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" event={"ID":"0b70f65a-9f12-458d-b942-61b7e178f019","Type":"ContainerStarted","Data":"57794ebbd10d8ffe32038896f5b55d94a02911329691e6c2b4a3fe4ddc02af9b"}
Nov 03 16:45:01.819550 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:01.819434    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" event={"ID":"0b70f65a-9f12-458d-b942-61b7e178f019","Type":"ContainerStarted","Data":"01cd62386cda5f3804f194e5905699ad6e497448c54adf9d9cf4857049774f04"}
Nov 03 16:45:01.936854 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:01.936801    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" podStartSLOduration=1.936784649 podStartE2EDuration="1.936784649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:45:01.936756245 +0000 UTC m=+1987.761207513" watchObservedRunningTime="2026-11-03 16:45:01.936784649 +0000 UTC m=+1987.761235921"
Nov 03 16:45:02.058516 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:02.058404    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" podStartSLOduration=2.058389678 podStartE2EDuration="2.058389678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:45:02.055669114 +0000 UTC m=+1987.880120385" watchObservedRunningTime="2026-11-03 16:45:02.058389678 +0000 UTC m=+1987.882840948"
Nov 03 16:45:02.829524 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:02.829048    5605 generic.go:356] "Generic (PLEG): container finished" podID="6747825c-9319-4280-8201-62063c3aa4f8" containerID="bfda005fb0a401fd84088204ada73336155403ddbff067dbff92ed29ec4843b2" exitCode=60
Nov 03 16:45:02.829524 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:02.829200    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" event={"ID":"6747825c-9319-4280-8201-62063c3aa4f8","Type":"ContainerDied","Data":"bfda005fb0a401fd84088204ada73336155403ddbff067dbff92ed29ec4843b2"}
Nov 03 16:45:02.829524 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:02.829477    5605 scope.go:122] "RemoveContainer" containerID="bfda005fb0a401fd84088204ada73336155403ddbff067dbff92ed29ec4843b2"
Nov 03 16:45:02.831185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:02.831160    5605 generic.go:356] "Generic (PLEG): container finished" podID="0b70f65a-9f12-458d-b942-61b7e178f019" containerID="57794ebbd10d8ffe32038896f5b55d94a02911329691e6c2b4a3fe4ddc02af9b" exitCode=60
Nov 03 16:45:02.831290 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:02.831281    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" event={"ID":"0b70f65a-9f12-458d-b942-61b7e178f019","Type":"ContainerDied","Data":"57794ebbd10d8ffe32038896f5b55d94a02911329691e6c2b4a3fe4ddc02af9b"}
Nov 03 16:45:02.831685 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:02.831607    5605 scope.go:122] "RemoveContainer" containerID="57794ebbd10d8ffe32038896f5b55d94a02911329691e6c2b4a3fe4ddc02af9b"
Nov 03 16:45:03.396940 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:03.396845    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:45:03.841484 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:03.841454    5605 generic.go:356] "Generic (PLEG): container finished" podID="6747825c-9319-4280-8201-62063c3aa4f8" containerID="61136bf6650a396549a19bb9c1d9eb91fc74bb8bdb0163f6b6a85a93a67306fd" exitCode=60
Nov 03 16:45:03.841986 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:03.841531    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" event={"ID":"6747825c-9319-4280-8201-62063c3aa4f8","Type":"ContainerDied","Data":"61136bf6650a396549a19bb9c1d9eb91fc74bb8bdb0163f6b6a85a93a67306fd"}
Nov 03 16:45:03.841986 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:03.841591    5605 scope.go:122] "RemoveContainer" containerID="bfda005fb0a401fd84088204ada73336155403ddbff067dbff92ed29ec4843b2"
Nov 03 16:45:03.841986 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:03.841952    5605 scope.go:122] "RemoveContainer" containerID="61136bf6650a396549a19bb9c1d9eb91fc74bb8bdb0163f6b6a85a93a67306fd"
Nov 03 16:45:03.842386 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:03.842362    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7_odh-ai-gateway-infra(6747825c-9319-4280-8201-62063c3aa4f8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" podUID="6747825c-9319-4280-8201-62063c3aa4f8"
Nov 03 16:45:03.843639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:03.843618    5605 generic.go:356] "Generic (PLEG): container finished" podID="0b70f65a-9f12-458d-b942-61b7e178f019" containerID="c25e4455007b9154c332d0f56467111f4f55a9d9bdef610c5086666b24669466" exitCode=60
Nov 03 16:45:03.843763 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:03.843697    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" event={"ID":"0b70f65a-9f12-458d-b942-61b7e178f019","Type":"ContainerDied","Data":"c25e4455007b9154c332d0f56467111f4f55a9d9bdef610c5086666b24669466"}
Nov 03 16:45:03.843975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:03.843962    5605 scope.go:122] "RemoveContainer" containerID="c25e4455007b9154c332d0f56467111f4f55a9d9bdef610c5086666b24669466"
Nov 03 16:45:03.844279 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:03.844259    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c_odh-ai-gateway-infra(0b70f65a-9f12-458d-b942-61b7e178f019)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" podUID="0b70f65a-9f12-458d-b942-61b7e178f019"
Nov 03 16:45:03.916680 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:03.916657    5605 scope.go:122] "RemoveContainer" containerID="57794ebbd10d8ffe32038896f5b55d94a02911329691e6c2b4a3fe4ddc02af9b"
Nov 03 16:45:04.853421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:04.853381    5605 scope.go:122] "RemoveContainer" containerID="61136bf6650a396549a19bb9c1d9eb91fc74bb8bdb0163f6b6a85a93a67306fd"
Nov 03 16:45:04.853898 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:04.853828    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7_odh-ai-gateway-infra(6747825c-9319-4280-8201-62063c3aa4f8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" podUID="6747825c-9319-4280-8201-62063c3aa4f8"
Nov 03 16:45:04.855027 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:04.855008    5605 scope.go:122] "RemoveContainer" containerID="c25e4455007b9154c332d0f56467111f4f55a9d9bdef610c5086666b24669466"
Nov 03 16:45:04.855330 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:04.855311    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c_odh-ai-gateway-infra(0b70f65a-9f12-458d-b942-61b7e178f019)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" podUID="0b70f65a-9f12-458d-b942-61b7e178f019"
Nov 03 16:45:17.301758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:17.301697    5605 scope.go:122] "RemoveContainer" containerID="61136bf6650a396549a19bb9c1d9eb91fc74bb8bdb0163f6b6a85a93a67306fd"
Nov 03 16:45:18.021965 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:18.021874    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" event={"ID":"6747825c-9319-4280-8201-62063c3aa4f8","Type":"ContainerStarted","Data":"688d91cde8b886e9267707ba8cf34ab2b55f8378c77f2d8d23ccfcb635786b7a"}
Nov 03 16:45:18.302420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:18.302389    5605 scope.go:122] "RemoveContainer" containerID="c25e4455007b9154c332d0f56467111f4f55a9d9bdef610c5086666b24669466"
Nov 03 16:45:19.030403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:19.030312    5605 generic.go:356] "Generic (PLEG): container finished" podID="6747825c-9319-4280-8201-62063c3aa4f8" containerID="688d91cde8b886e9267707ba8cf34ab2b55f8378c77f2d8d23ccfcb635786b7a" exitCode=60
Nov 03 16:45:19.030403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:19.030387    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" event={"ID":"6747825c-9319-4280-8201-62063c3aa4f8","Type":"ContainerDied","Data":"688d91cde8b886e9267707ba8cf34ab2b55f8378c77f2d8d23ccfcb635786b7a"}
Nov 03 16:45:19.030630 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:19.030455    5605 scope.go:122] "RemoveContainer" containerID="61136bf6650a396549a19bb9c1d9eb91fc74bb8bdb0163f6b6a85a93a67306fd"
Nov 03 16:45:19.030914 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:19.030888    5605 scope.go:122] "RemoveContainer" containerID="688d91cde8b886e9267707ba8cf34ab2b55f8378c77f2d8d23ccfcb635786b7a"
Nov 03 16:45:19.031276 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:19.031250    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7_odh-ai-gateway-infra(6747825c-9319-4280-8201-62063c3aa4f8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" podUID="6747825c-9319-4280-8201-62063c3aa4f8"
Nov 03 16:45:19.032512 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:19.032491    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" event={"ID":"0b70f65a-9f12-458d-b942-61b7e178f019","Type":"ContainerStarted","Data":"b5169106a9b77c02b0d9ba292376589c744d0d5fe1957aff26e8ca1be56cdfc7"}
Nov 03 16:45:19.162360 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:19.162324    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"]
Nov 03 16:45:20.043028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.042997    5605 generic.go:356] "Generic (PLEG): container finished" podID="0b70f65a-9f12-458d-b942-61b7e178f019" containerID="b5169106a9b77c02b0d9ba292376589c744d0d5fe1957aff26e8ca1be56cdfc7" exitCode=60
Nov 03 16:45:20.043373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.043075    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" event={"ID":"0b70f65a-9f12-458d-b942-61b7e178f019","Type":"ContainerDied","Data":"b5169106a9b77c02b0d9ba292376589c744d0d5fe1957aff26e8ca1be56cdfc7"}
Nov 03 16:45:20.043373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.043140    5605 scope.go:122] "RemoveContainer" containerID="c25e4455007b9154c332d0f56467111f4f55a9d9bdef610c5086666b24669466"
Nov 03 16:45:20.043584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.043569    5605 scope.go:122] "RemoveContainer" containerID="b5169106a9b77c02b0d9ba292376589c744d0d5fe1957aff26e8ca1be56cdfc7"
Nov 03 16:45:20.043979 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:20.043958    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c_odh-ai-gateway-infra(0b70f65a-9f12-458d-b942-61b7e178f019)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" podUID="0b70f65a-9f12-458d-b942-61b7e178f019"
Nov 03 16:45:20.221561 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.221537    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"
Nov 03 16:45:20.312517 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.312446    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"]
Nov 03 16:45:20.334230 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.334206    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6747825c-9319-4280-8201-62063c3aa4f8-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6747825c-9319-4280-8201-62063c3aa4f8-maas-api-service-ca\") pod \"6747825c-9319-4280-8201-62063c3aa4f8\" (UID: \"6747825c-9319-4280-8201-62063c3aa4f8\") "
Nov 03 16:45:20.334395 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.334295    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6747825c-9319-4280-8201-62063c3aa4f8-kube-api-access-8c8dv\" (UniqueName: \"kubernetes.io/projected/6747825c-9319-4280-8201-62063c3aa4f8-kube-api-access-8c8dv\") pod \"6747825c-9319-4280-8201-62063c3aa4f8\" (UID: \"6747825c-9319-4280-8201-62063c3aa4f8\") "
Nov 03 16:45:20.334645 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.334617    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6747825c-9319-4280-8201-62063c3aa4f8-maas-api-service-ca" pod "6747825c-9319-4280-8201-62063c3aa4f8" (UID: "6747825c-9319-4280-8201-62063c3aa4f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:45:20.335146 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.335126    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6747825c-9319-4280-8201-62063c3aa4f8-maas-api-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:20.336586 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.336560    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6747825c-9319-4280-8201-62063c3aa4f8-kube-api-access-8c8dv" pod "6747825c-9319-4280-8201-62063c3aa4f8" (UID: "6747825c-9319-4280-8201-62063c3aa4f8"). InnerVolumeSpecName "kube-api-access-8c8dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:45:20.436918 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:20.436887    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8c8dv\" (UniqueName: \"kubernetes.io/projected/6747825c-9319-4280-8201-62063c3aa4f8-kube-api-access-8c8dv\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:21.051543 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.051505    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"
Nov 03 16:45:21.051543 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.051526    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7" event={"ID":"6747825c-9319-4280-8201-62063c3aa4f8","Type":"ContainerDied","Data":"abc105dc2ea5111e42a20de5ace98e394fdf0e7381df763b29ee6c917a6a5a52"}
Nov 03 16:45:21.052158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.051599    5605 scope.go:122] "RemoveContainer" containerID="688d91cde8b886e9267707ba8cf34ab2b55f8378c77f2d8d23ccfcb635786b7a"
Nov 03 16:45:21.207978 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.207948    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"
Nov 03 16:45:21.228708 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.228677    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"]
Nov 03 16:45:21.246489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.246454    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b70f65a-9f12-458d-b942-61b7e178f019-kube-api-access-l4wrg\" (UniqueName: \"kubernetes.io/projected/0b70f65a-9f12-458d-b942-61b7e178f019-kube-api-access-l4wrg\") pod \"0b70f65a-9f12-458d-b942-61b7e178f019\" (UID: \"0b70f65a-9f12-458d-b942-61b7e178f019\") "
Nov 03 16:45:21.246670 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.246652    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b70f65a-9f12-458d-b942-61b7e178f019-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b70f65a-9f12-458d-b942-61b7e178f019-maas-api-service-ca\") pod \"0b70f65a-9f12-458d-b942-61b7e178f019\" (UID: \"0b70f65a-9f12-458d-b942-61b7e178f019\") "
Nov 03 16:45:21.247021 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.246999    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b70f65a-9f12-458d-b942-61b7e178f019-maas-api-service-ca" pod "0b70f65a-9f12-458d-b942-61b7e178f019" (UID: "0b70f65a-9f12-458d-b942-61b7e178f019"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:45:21.247286 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.247267    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b70f65a-9f12-458d-b942-61b7e178f019-maas-api-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:21.248968 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.248941    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b70f65a-9f12-458d-b942-61b7e178f019-kube-api-access-l4wrg" pod "0b70f65a-9f12-458d-b942-61b7e178f019" (UID: "0b70f65a-9f12-458d-b942-61b7e178f019"). InnerVolumeSpecName "kube-api-access-l4wrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:45:21.280197 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.280173    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-42da0f-29895405-gjnk7"]
Nov 03 16:45:21.349111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.349069    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4wrg\" (UniqueName: \"kubernetes.io/projected/0b70f65a-9f12-458d-b942-61b7e178f019-kube-api-access-l4wrg\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:21.556135 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.556095    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"]
Nov 03 16:45:21.556321 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.556294    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm" podUID="02b89011-2c14-4641-86dd-6f05a6df2c33" containerName="maas-api" containerID="cri-o://8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce" gracePeriod=30
Nov 03 16:45:21.911114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.911089    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:45:21.962588 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.962492    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02b89011-2c14-4641-86dd-6f05a6df2c33-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02b89011-2c14-4641-86dd-6f05a6df2c33-openshift-service-ca\") pod \"02b89011-2c14-4641-86dd-6f05a6df2c33\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") "
Nov 03 16:45:21.962588 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.962543    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-maas-api-tls\") pod \"02b89011-2c14-4641-86dd-6f05a6df2c33\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") "
Nov 03 16:45:21.962881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.962654    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02b89011-2c14-4641-86dd-6f05a6df2c33-kube-api-access-s2lfg\" (UniqueName: \"kubernetes.io/projected/02b89011-2c14-4641-86dd-6f05a6df2c33-kube-api-access-s2lfg\") pod \"02b89011-2c14-4641-86dd-6f05a6df2c33\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") "
Nov 03 16:45:21.962881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.962763    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-metrics-certs\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-metrics-certs\") pod \"02b89011-2c14-4641-86dd-6f05a6df2c33\" (UID: \"02b89011-2c14-4641-86dd-6f05a6df2c33\") "
Nov 03 16:45:21.963035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.962997    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02b89011-2c14-4641-86dd-6f05a6df2c33-openshift-service-ca" pod "02b89011-2c14-4641-86dd-6f05a6df2c33" (UID: "02b89011-2c14-4641-86dd-6f05a6df2c33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:45:21.964050 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.964023    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02b89011-2c14-4641-86dd-6f05a6df2c33-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:21.965893 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.965864    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b89011-2c14-4641-86dd-6f05a6df2c33-kube-api-access-s2lfg" pod "02b89011-2c14-4641-86dd-6f05a6df2c33" (UID: "02b89011-2c14-4641-86dd-6f05a6df2c33"). InnerVolumeSpecName "kube-api-access-s2lfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:45:21.965980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.965931    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-maas-api-tls" pod "02b89011-2c14-4641-86dd-6f05a6df2c33" (UID: "02b89011-2c14-4641-86dd-6f05a6df2c33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:45:21.966663 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:21.966643    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-metrics-certs" pod "02b89011-2c14-4641-86dd-6f05a6df2c33" (UID: "02b89011-2c14-4641-86dd-6f05a6df2c33"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:45:22.077752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.075367    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2lfg\" (UniqueName: \"kubernetes.io/projected/02b89011-2c14-4641-86dd-6f05a6df2c33-kube-api-access-s2lfg\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:22.077752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.075410    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:22.077752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.075425    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02b89011-2c14-4641-86dd-6f05a6df2c33-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:22.135581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.135544    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"
Nov 03 16:45:22.135581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.135561    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c" event={"ID":"0b70f65a-9f12-458d-b942-61b7e178f019","Type":"ContainerDied","Data":"01cd62386cda5f3804f194e5905699ad6e497448c54adf9d9cf4857049774f04"}
Nov 03 16:45:22.135883 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.135631    5605 scope.go:122] "RemoveContainer" containerID="b5169106a9b77c02b0d9ba292376589c744d0d5fe1957aff26e8ca1be56cdfc7"
Nov 03 16:45:22.139211 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.139180    5605 generic.go:356] "Generic (PLEG): container finished" podID="02b89011-2c14-4641-86dd-6f05a6df2c33" containerID="8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce" exitCode=0
Nov 03 16:45:22.139344 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.139264    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm" event={"ID":"02b89011-2c14-4641-86dd-6f05a6df2c33","Type":"ContainerDied","Data":"8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce"}
Nov 03 16:45:22.139344 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.139310    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm" event={"ID":"02b89011-2c14-4641-86dd-6f05a6df2c33","Type":"ContainerDied","Data":"2274caab63e6e4403a83daf8292f0275413ce37670285186967a8eba1a500a30"}
Nov 03 16:45:22.139471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.139390    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"
Nov 03 16:45:22.152995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.152925    5605 scope.go:122] "RemoveContainer" containerID="8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce"
Nov 03 16:45:22.171064 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.166411    5605 scope.go:122] "RemoveContainer" containerID="8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce"
Nov 03 16:45:22.171064 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:22.166837    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce\": container with ID starting with 8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce not found: ID does not exist" containerID="8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce"
Nov 03 16:45:22.171064 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.166875    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce"} err="failed to get container status \"8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce\": rpc error: code = NotFound desc = could not find container \"8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce\": container with ID starting with 8dcbaea76b88dc857d28e83632fbd42c176358a2a027fd158332472e22d4b4ce not found: ID does not exist"
Nov 03 16:45:22.312452 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.312364    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6747825c-9319-4280-8201-62063c3aa4f8" path="/var/lib/kubelet/pods/6747825c-9319-4280-8201-62063c3aa4f8/volumes"
Nov 03 16:45:22.312785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.312768    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"]
Nov 03 16:45:22.316455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.316429    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-438248-74f685875d-fkjnm"]
Nov 03 16:45:22.434585 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.434539    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"]
Nov 03 16:45:22.440575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:22.440545    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-195095-29895405-87t2c"]
Nov 03 16:45:24.311303 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:24.311262    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02b89011-2c14-4641-86dd-6f05a6df2c33" path="/var/lib/kubelet/pods/02b89011-2c14-4641-86dd-6f05a6df2c33/volumes"
Nov 03 16:45:24.311702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:24.311686    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b70f65a-9f12-458d-b942-61b7e178f019" path="/var/lib/kubelet/pods/0b70f65a-9f12-458d-b942-61b7e178f019/volumes"
Nov 03 16:45:33.040257 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:33.040178    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"]
Nov 03 16:45:33.040877 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:33.040416    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy" containerID="cri-o://eb83b42d19b9983fcd1444a1b0d1178fd5a9c669fbae84ef7e4e6dee60b63e16" gracePeriod=30
Nov 03 16:45:33.397524 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:33.397488    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:45:37.194041 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:37.193998    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"]
Nov 03 16:45:37.311222 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:37.311185    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"]
Nov 03 16:45:38.284879 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.284842    5605 generic.go:356] "Generic (PLEG): container finished" podID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerID="eb83b42d19b9983fcd1444a1b0d1178fd5a9c669fbae84ef7e4e6dee60b63e16" exitCode=0
Nov 03 16:45:38.285302 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.284895    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" event={"ID":"84b165c2-3d89-452c-852b-ffa8ca079c72","Type":"ContainerDied","Data":"eb83b42d19b9983fcd1444a1b0d1178fd5a9c669fbae84ef7e4e6dee60b63e16"}
Nov 03 16:45:38.285302 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.284953    5605 scope.go:122] "RemoveContainer" containerID="1a56961bfa843ddf73915395d367b9cda57b21030184ddfc61d097f2a3fdc630"
Nov 03 16:45:38.353765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.353716    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:45:38.513400 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.513364    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-certs\") pod \"84b165c2-3d89-452c-852b-ffa8ca079c72\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") "
Nov 03 16:45:38.513603 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.513451    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-podinfo\") pod \"84b165c2-3d89-452c-852b-ffa8ca079c72\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") "
Nov 03 16:45:38.513666 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.513604    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-credential-socket\") pod \"84b165c2-3d89-452c-852b-ffa8ca079c72\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") "
Nov 03 16:45:38.513666 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.513638    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-token\" (UniqueName: \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-token\") pod \"84b165c2-3d89-452c-852b-ffa8ca079c72\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") "
Nov 03 16:45:38.513800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.513691    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-kube-api-access-jpvft\" (UniqueName: \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-kube-api-access-jpvft\") pod \"84b165c2-3d89-452c-852b-ffa8ca079c72\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") "
Nov 03 16:45:38.513800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.513693    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-certs" pod "84b165c2-3d89-452c-852b-ffa8ca079c72" (UID: "84b165c2-3d89-452c-852b-ffa8ca079c72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:45:38.513893 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.513816    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84b165c2-3d89-452c-852b-ffa8ca079c72-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84b165c2-3d89-452c-852b-ffa8ca079c72-istiod-ca-cert\") pod \"84b165c2-3d89-452c-852b-ffa8ca079c72\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") "
Nov 03 16:45:38.513893 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.513858    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-data\") pod \"84b165c2-3d89-452c-852b-ffa8ca079c72\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") "
Nov 03 16:45:38.513980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.513916    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-socket\") pod \"84b165c2-3d89-452c-852b-ffa8ca079c72\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") "
Nov 03 16:45:38.513980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.513942    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-envoy\") pod \"84b165c2-3d89-452c-852b-ffa8ca079c72\" (UID: \"84b165c2-3d89-452c-852b-ffa8ca079c72\") "
Nov 03 16:45:38.514251 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.514219    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-credential-socket" pod "84b165c2-3d89-452c-852b-ffa8ca079c72" (UID: "84b165c2-3d89-452c-852b-ffa8ca079c72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:45:38.514422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.514325    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-data" pod "84b165c2-3d89-452c-852b-ffa8ca079c72" (UID: "84b165c2-3d89-452c-852b-ffa8ca079c72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:45:38.514699 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.514681    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-socket" pod "84b165c2-3d89-452c-852b-ffa8ca079c72" (UID: "84b165c2-3d89-452c-852b-ffa8ca079c72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:45:38.514848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.514707    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84b165c2-3d89-452c-852b-ffa8ca079c72-istiod-ca-cert" pod "84b165c2-3d89-452c-852b-ffa8ca079c72" (UID: "84b165c2-3d89-452c-852b-ffa8ca079c72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:45:38.514848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.514784    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:38.514978 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.514860    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:38.514978 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.514876    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:38.516471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.516445    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-token" pod "84b165c2-3d89-452c-852b-ffa8ca079c72" (UID: "84b165c2-3d89-452c-852b-ffa8ca079c72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:45:38.516606 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.516477    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-podinfo" pod "84b165c2-3d89-452c-852b-ffa8ca079c72" (UID: "84b165c2-3d89-452c-852b-ffa8ca079c72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:45:38.517114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.517082    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-envoy" pod "84b165c2-3d89-452c-852b-ffa8ca079c72" (UID: "84b165c2-3d89-452c-852b-ffa8ca079c72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:45:38.517220 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.517199    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-kube-api-access-jpvft" pod "84b165c2-3d89-452c-852b-ffa8ca079c72" (UID: "84b165c2-3d89-452c-852b-ffa8ca079c72"). InnerVolumeSpecName "kube-api-access-jpvft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:45:38.616702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.616670    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:38.616702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.616704    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:38.616934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.616720    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpvft\" (UniqueName: \"kubernetes.io/projected/84b165c2-3d89-452c-852b-ffa8ca079c72-kube-api-access-jpvft\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:38.616934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.616756    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84b165c2-3d89-452c-852b-ffa8ca079c72-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:38.616934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.616769    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:38.616934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:38.616778    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84b165c2-3d89-452c-852b-ffa8ca079c72-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:39.292884 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:39.292857    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"
Nov 03 16:45:39.293307 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:39.292901    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv" event={"ID":"84b165c2-3d89-452c-852b-ffa8ca079c72","Type":"ContainerDied","Data":"b3ff156ac4bbbf969cb21db105fa0f1cfaa72c83e796b7a1e57551987f278405"}
Nov 03 16:45:39.293307 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:39.292957    5605 scope.go:122] "RemoveContainer" containerID="eb83b42d19b9983fcd1444a1b0d1178fd5a9c669fbae84ef7e4e6dee60b63e16"
Nov 03 16:45:39.528323 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:39.527009    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"]
Nov 03 16:45:39.570899 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:39.570820    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-438248-openshift-default-c95bf5b47-hfplv"]
Nov 03 16:45:40.310780 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:40.310745    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" path="/var/lib/kubelet/pods/84b165c2-3d89-452c-852b-ffa8ca079c72/volumes"
Nov 03 16:45:42.195028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.194987    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n" podUID="f315d524-c3e5-405a-85e6-e7ca8d2718cf" containerName="payload-processing" containerID="cri-o://0d73d8a9bf92e35ab3d672dbc1ce673417704405922d549d0c0382e52a9c1909" gracePeriod=25
Nov 03 16:45:42.311522 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.311467    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b" podUID="c77fe985-3f03-4e0f-b4fe-325d2d3fb79d" containerName="payload-pre-processing" containerID="cri-o://d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac" gracePeriod=25
Nov 03 16:45:42.318108 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.318072    5605 generic.go:356] "Generic (PLEG): container finished" podID="f315d524-c3e5-405a-85e6-e7ca8d2718cf" containerID="0d73d8a9bf92e35ab3d672dbc1ce673417704405922d549d0c0382e52a9c1909" exitCode=0
Nov 03 16:45:42.318205 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.318139    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n" event={"ID":"f315d524-c3e5-405a-85e6-e7ca8d2718cf","Type":"ContainerDied","Data":"0d73d8a9bf92e35ab3d672dbc1ce673417704405922d549d0c0382e52a9c1909"}
Nov 03 16:45:42.499904 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.499870    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:45:42.574272 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.574240    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:45:42.610437 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.610407    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-kube-api-access-t2pk5\" (UniqueName: \"kubernetes.io/projected/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-kube-api-access-t2pk5\") pod \"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d\" (UID: \"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d\") "
Nov 03 16:45:42.610632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.610479    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f315d524-c3e5-405a-85e6-e7ca8d2718cf-kube-api-access-6x2fr\" (UniqueName: \"kubernetes.io/projected/f315d524-c3e5-405a-85e6-e7ca8d2718cf-kube-api-access-6x2fr\") pod \"f315d524-c3e5-405a-85e6-e7ca8d2718cf\" (UID: \"f315d524-c3e5-405a-85e6-e7ca8d2718cf\") "
Nov 03 16:45:42.610632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.610596    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f315d524-c3e5-405a-85e6-e7ca8d2718cf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f315d524-c3e5-405a-85e6-e7ca8d2718cf-plugins-config-volume\") pod \"f315d524-c3e5-405a-85e6-e7ca8d2718cf\" (UID: \"f315d524-c3e5-405a-85e6-e7ca8d2718cf\") "
Nov 03 16:45:42.610632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.610624    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-plugins-config-volume\") pod \"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d\" (UID: \"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d\") "
Nov 03 16:45:42.611037 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.611007    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f315d524-c3e5-405a-85e6-e7ca8d2718cf-plugins-config-volume" pod "f315d524-c3e5-405a-85e6-e7ca8d2718cf" (UID: "f315d524-c3e5-405a-85e6-e7ca8d2718cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:45:42.611037 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.611027    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-plugins-config-volume" pod "c77fe985-3f03-4e0f-b4fe-325d2d3fb79d" (UID: "c77fe985-3f03-4e0f-b4fe-325d2d3fb79d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:45:42.611560 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.611539    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:42.611616 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.611569    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f315d524-c3e5-405a-85e6-e7ca8d2718cf-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:42.612856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.612834    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f315d524-c3e5-405a-85e6-e7ca8d2718cf-kube-api-access-6x2fr" pod "f315d524-c3e5-405a-85e6-e7ca8d2718cf" (UID: "f315d524-c3e5-405a-85e6-e7ca8d2718cf"). InnerVolumeSpecName "kube-api-access-6x2fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:45:42.612965 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.612915    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-kube-api-access-t2pk5" pod "c77fe985-3f03-4e0f-b4fe-325d2d3fb79d" (UID: "c77fe985-3f03-4e0f-b4fe-325d2d3fb79d"). InnerVolumeSpecName "kube-api-access-t2pk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:45:42.712599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.712562    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2pk5\" (UniqueName: \"kubernetes.io/projected/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d-kube-api-access-t2pk5\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:42.712599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:42.712594    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6x2fr\" (UniqueName: \"kubernetes.io/projected/f315d524-c3e5-405a-85e6-e7ca8d2718cf-kube-api-access-6x2fr\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:43.327091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.327047    5605 generic.go:356] "Generic (PLEG): container finished" podID="c77fe985-3f03-4e0f-b4fe-325d2d3fb79d" containerID="d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac" exitCode=0
Nov 03 16:45:43.327578 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.327103    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"
Nov 03 16:45:43.327578 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.327126    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b" event={"ID":"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d","Type":"ContainerDied","Data":"d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac"}
Nov 03 16:45:43.327578 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.327181    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b" event={"ID":"c77fe985-3f03-4e0f-b4fe-325d2d3fb79d","Type":"ContainerDied","Data":"0b06dec5fb3b7f3cc778f65bffc3756f0d5eac6b812aa725e37e6c598ea434b4"}
Nov 03 16:45:43.327578 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.327204    5605 scope.go:122] "RemoveContainer" containerID="d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac"
Nov 03 16:45:43.328926 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.328901    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n" event={"ID":"f315d524-c3e5-405a-85e6-e7ca8d2718cf","Type":"ContainerDied","Data":"115a892f06fd65b755cf18d400f914df4a75bd9dd46eb461e2acf6b83127e70f"}
Nov 03 16:45:43.329050 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.328932    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"
Nov 03 16:45:43.338976 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.338608    5605 scope.go:122] "RemoveContainer" containerID="d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac"
Nov 03 16:45:43.339071 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:43.338972    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac\": container with ID starting with d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac not found: ID does not exist" containerID="d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac"
Nov 03 16:45:43.339071 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.338997    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac"} err="failed to get container status \"d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac\": rpc error: code = NotFound desc = could not find container \"d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac\": container with ID starting with d269991cf31abfc3ea59dc7d29a96c2eaf641a6469e2a35f48db448a0626d8ac not found: ID does not exist"
Nov 03 16:45:43.339071 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.339013    5605 scope.go:122] "RemoveContainer" containerID="0d73d8a9bf92e35ab3d672dbc1ce673417704405922d549d0c0382e52a9c1909"
Nov 03 16:45:43.500351 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.500271    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"]
Nov 03 16:45:43.551316 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.551288    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cc4dcd-f898d4ffc-gnx7n"]
Nov 03 16:45:43.621997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.621935    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"]
Nov 03 16:45:43.634159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:43.634118    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cc4dcd-587d9955cf-m2h6b"]
Nov 03 16:45:44.310431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.310395    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c77fe985-3f03-4e0f-b4fe-325d2d3fb79d" path="/var/lib/kubelet/pods/c77fe985-3f03-4e0f-b4fe-325d2d3fb79d/volumes"
Nov 03 16:45:44.311004 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.310981    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f315d524-c3e5-405a-85e6-e7ca8d2718cf" path="/var/lib/kubelet/pods/f315d524-c3e5-405a-85e6-e7ca8d2718cf/volumes"
Nov 03 16:45:44.364949 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.364916    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"]
Nov 03 16:45:44.365551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365535    5605 cpu_manager.go:414] "containerMap: removing container" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy"
Nov 03 16:45:44.365593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365554    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy"
Nov 03 16:45:44.365593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365569    5605 cpu_manager.go:414] "containerMap: removing container" podUID="0b70f65a-9f12-458d-b942-61b7e178f019" containerName="cleanup"
Nov 03 16:45:44.365593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365577    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b70f65a-9f12-458d-b942-61b7e178f019" containerName="cleanup"
Nov 03 16:45:44.365593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365588    5605 cpu_manager.go:414] "containerMap: removing container" podUID="0b70f65a-9f12-458d-b942-61b7e178f019" containerName="cleanup"
Nov 03 16:45:44.365593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365593    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b70f65a-9f12-458d-b942-61b7e178f019" containerName="cleanup"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365607    5605 cpu_manager.go:414] "containerMap: removing container" podUID="c77fe985-3f03-4e0f-b4fe-325d2d3fb79d" containerName="payload-pre-processing"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365612    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c77fe985-3f03-4e0f-b4fe-325d2d3fb79d" containerName="payload-pre-processing"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365624    5605 cpu_manager.go:414] "containerMap: removing container" podUID="02b89011-2c14-4641-86dd-6f05a6df2c33" containerName="maas-api"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365628    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02b89011-2c14-4641-86dd-6f05a6df2c33" containerName="maas-api"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365642    5605 cpu_manager.go:414] "containerMap: removing container" podUID="6747825c-9319-4280-8201-62063c3aa4f8" containerName="cleanup"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365647    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6747825c-9319-4280-8201-62063c3aa4f8" containerName="cleanup"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365656    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f315d524-c3e5-405a-85e6-e7ca8d2718cf" containerName="payload-processing"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365661    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f315d524-c3e5-405a-85e6-e7ca8d2718cf" containerName="payload-processing"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365680    5605 cpu_manager.go:414] "containerMap: removing container" podUID="6747825c-9319-4280-8201-62063c3aa4f8" containerName="cleanup"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365684    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6747825c-9319-4280-8201-62063c3aa4f8" containerName="cleanup"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365691    5605 cpu_manager.go:414] "containerMap: removing container" podUID="6747825c-9319-4280-8201-62063c3aa4f8" containerName="cleanup"
Nov 03 16:45:44.365760 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365696    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6747825c-9319-4280-8201-62063c3aa4f8" containerName="cleanup"
Nov 03 16:45:44.366112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365817    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy"
Nov 03 16:45:44.366112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365825    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b70f65a-9f12-458d-b942-61b7e178f019" containerName="cleanup"
Nov 03 16:45:44.366112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365834    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b70f65a-9f12-458d-b942-61b7e178f019" containerName="cleanup"
Nov 03 16:45:44.366112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365842    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="c77fe985-3f03-4e0f-b4fe-325d2d3fb79d" containerName="payload-pre-processing"
Nov 03 16:45:44.366112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365850    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="02b89011-2c14-4641-86dd-6f05a6df2c33" containerName="maas-api"
Nov 03 16:45:44.366112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365858    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="6747825c-9319-4280-8201-62063c3aa4f8" containerName="cleanup"
Nov 03 16:45:44.366112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365863    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f315d524-c3e5-405a-85e6-e7ca8d2718cf" containerName="payload-processing"
Nov 03 16:45:44.366112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365870    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy"
Nov 03 16:45:44.366112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365877    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="6747825c-9319-4280-8201-62063c3aa4f8" containerName="cleanup"
Nov 03 16:45:44.366112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.365883    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="6747825c-9319-4280-8201-62063c3aa4f8" containerName="cleanup"
Nov 03 16:45:44.370717 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.370698    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.426762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.426720    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"]
Nov 03 16:45:44.439178 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.439149    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-maas-api-tls\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.439317 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.439207    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-openshift-service-ca\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.439493 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.439457    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-metrics-certs\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.439648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.439627    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4jh2\" (UniqueName: \"kubernetes.io/projected/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-kube-api-access-h4jh2\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.474780 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.474755    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6f58cf-serving-cert\""
Nov 03 16:45:44.541173 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.541142    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-metrics-certs\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.541345 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.541246    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4jh2\" (UniqueName: \"kubernetes.io/projected/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-kube-api-access-h4jh2\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.541345 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.541296    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-maas-api-tls\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.541345 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.541322    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-openshift-service-ca\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.541510 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:44.541401    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-serving-cert: secret "maas-api-e2e-api-a-6f58cf-serving-cert" not found
Nov 03 16:45:44.541510 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:44.541488    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-maas-api-tls podName:16e5e2dc-9bb6-4120-9904-06e1e2da1f8b nodeName:}" failed. No retries permitted until 2026-11-03 16:45:45.041466533 +0000 UTC m=+2030.865917793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-maas-api-tls") pod "maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb" (UID: "16e5e2dc-9bb6-4120-9904-06e1e2da1f8b") : secret "maas-api-e2e-api-a-6f58cf-serving-cert" not found
Nov 03 16:45:44.542111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.542090    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-openshift-service-ca\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.543698 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.543676    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-metrics-certs\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:44.603254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:44.603222    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4jh2\" (UniqueName: \"kubernetes.io/projected/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-kube-api-access-h4jh2\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:45.076927 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:45.076836    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-maas-api-tls\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:45.079705 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:45.079674    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-maas-api-tls\") pod \"maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:45.286076 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:45.286043    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:45.683905 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:45.683876    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"]
Nov 03 16:45:45.684705 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:45:45.684672    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e5e2dc_9bb6_4120_9904_06e1e2da1f8b.slice/crio-2f030d3b445971faad2a463ea7d35b180713b0c6041c02f922d3a12a7a2b3b87 WatchSource:0}: Error finding container 2f030d3b445971faad2a463ea7d35b180713b0c6041c02f922d3a12a7a2b3b87: Status 404 returned error can't find the container with id 2f030d3b445971faad2a463ea7d35b180713b0c6041c02f922d3a12a7a2b3b87
Nov 03 16:45:46.357510 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:46.357477    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb" event={"ID":"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b","Type":"ContainerStarted","Data":"2f030d3b445971faad2a463ea7d35b180713b0c6041c02f922d3a12a7a2b3b87"}
Nov 03 16:45:47.366083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:47.366044    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb" event={"ID":"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b","Type":"ContainerStarted","Data":"bb20d2b5ddb5bb171d1b6c12f52062c735a805e465bbce6b55f4b3aa458d60b2"}
Nov 03 16:45:47.366553 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:47.366268    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:47.524006 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:47.523956    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb" podStartSLOduration=2.665460978 podStartE2EDuration="3.523943742s" totalImagesPullingTime="858.482764ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:45:47.52359104 +0000 UTC m=+2033.348042310" watchObservedRunningTime="2026-11-03 16:45:47.523943742 +0000 UTC m=+2033.348395013"
Nov 03 16:45:48.908197 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:48.908161    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"]
Nov 03 16:45:49.018434 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:49.018388    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"]
Nov 03 16:45:53.379892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:53.379859    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:45:53.908529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:53.908489    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9" podUID="54734e4d-d0a1-4a3a-a617-fbffc691d3e5" containerName="payload-processing" containerID="cri-o://0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e" gracePeriod=25
Nov 03 16:45:54.014257 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.014226    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"]
Nov 03 16:45:54.014446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.014426    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2" podUID="f14c005c-83b1-4d64-9195-c52539ff7c35" containerName="maas-api" containerID="cri-o://a0455e0331c2c9c94ff5de9b82706f034fdab0c86e10d29ac9e7ca17f222a893" gracePeriod=30
Nov 03 16:45:54.019033 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.018963    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst" podUID="814d68eb-8cb5-4eea-8c2a-006a6e8b80e7" containerName="payload-pre-processing" containerID="cri-o://af5f542bf6b808a100df23796a0fcd2a746b2e57a52233fca64abff48416562c" gracePeriod=25
Nov 03 16:45:54.331772 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.331717    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:45:54.481805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.473214    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-plugins-config-volume\") pod \"54734e4d-d0a1-4a3a-a617-fbffc691d3e5\" (UID: \"54734e4d-d0a1-4a3a-a617-fbffc691d3e5\") "
Nov 03 16:45:54.481805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.473290    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-kube-api-access-x88ll\" (UniqueName: \"kubernetes.io/projected/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-kube-api-access-x88ll\") pod \"54734e4d-d0a1-4a3a-a617-fbffc691d3e5\" (UID: \"54734e4d-d0a1-4a3a-a617-fbffc691d3e5\") "
Nov 03 16:45:54.481805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.473757    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-plugins-config-volume" pod "54734e4d-d0a1-4a3a-a617-fbffc691d3e5" (UID: "54734e4d-d0a1-4a3a-a617-fbffc691d3e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:45:54.481805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.474325    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:54.534840 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.529147    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-kube-api-access-x88ll" pod "54734e4d-d0a1-4a3a-a617-fbffc691d3e5" (UID: "54734e4d-d0a1-4a3a-a617-fbffc691d3e5"). InnerVolumeSpecName "kube-api-access-x88ll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:45:54.555376 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.542452    5605 generic.go:356] "Generic (PLEG): container finished" podID="f14c005c-83b1-4d64-9195-c52539ff7c35" containerID="a0455e0331c2c9c94ff5de9b82706f034fdab0c86e10d29ac9e7ca17f222a893" exitCode=0
Nov 03 16:45:54.555376 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.542623    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2" event={"ID":"f14c005c-83b1-4d64-9195-c52539ff7c35","Type":"ContainerDied","Data":"a0455e0331c2c9c94ff5de9b82706f034fdab0c86e10d29ac9e7ca17f222a893"}
Nov 03 16:45:54.557894 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.557606    5605 generic.go:356] "Generic (PLEG): container finished" podID="814d68eb-8cb5-4eea-8c2a-006a6e8b80e7" containerID="af5f542bf6b808a100df23796a0fcd2a746b2e57a52233fca64abff48416562c" exitCode=0
Nov 03 16:45:54.558047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.557916    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst" event={"ID":"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7","Type":"ContainerDied","Data":"af5f542bf6b808a100df23796a0fcd2a746b2e57a52233fca64abff48416562c"}
Nov 03 16:45:54.562855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.560502    5605 generic.go:356] "Generic (PLEG): container finished" podID="54734e4d-d0a1-4a3a-a617-fbffc691d3e5" containerID="0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e" exitCode=0
Nov 03 16:45:54.562855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.560593    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9" event={"ID":"54734e4d-d0a1-4a3a-a617-fbffc691d3e5","Type":"ContainerDied","Data":"0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e"}
Nov 03 16:45:54.562855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.560637    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9" event={"ID":"54734e4d-d0a1-4a3a-a617-fbffc691d3e5","Type":"ContainerDied","Data":"4b5a934f9088f3a6f0ca3e2a61ce7a55396f22aa1dcaa2a1b97fd4305d299861"}
Nov 03 16:45:54.562855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.560661    5605 scope.go:122] "RemoveContainer" containerID="0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e"
Nov 03 16:45:54.562855 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.560911    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"
Nov 03 16:45:54.572851 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.572827    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:45:54.575504 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.575481    5605 scope.go:122] "RemoveContainer" containerID="0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e"
Nov 03 16:45:54.575827 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:45:54.575802    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e\": container with ID starting with 0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e not found: ID does not exist" containerID="0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e"
Nov 03 16:45:54.575907 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.575836    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e"} err="failed to get container status \"0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e\": rpc error: code = NotFound desc = could not find container \"0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e\": container with ID starting with 0dbc101cf1cf86307f485bd94f6b9f8495eafc6194045030d4d393b4cd160f3e not found: ID does not exist"
Nov 03 16:45:54.576257 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.576234    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x88ll\" (UniqueName: \"kubernetes.io/projected/54734e4d-d0a1-4a3a-a617-fbffc691d3e5-kube-api-access-x88ll\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:54.616593 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.616566    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:45:54.676966 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.676932    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f14c005c-83b1-4d64-9195-c52539ff7c35-kube-api-access-sjmkz\" (UniqueName: \"kubernetes.io/projected/f14c005c-83b1-4d64-9195-c52539ff7c35-kube-api-access-sjmkz\") pod \"f14c005c-83b1-4d64-9195-c52539ff7c35\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") "
Nov 03 16:45:54.677134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.677041    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-metrics-certs\") pod \"f14c005c-83b1-4d64-9195-c52539ff7c35\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") "
Nov 03 16:45:54.677134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.677079    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-maas-api-tls\") pod \"f14c005c-83b1-4d64-9195-c52539ff7c35\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") "
Nov 03 16:45:54.677221 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.677163    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f14c005c-83b1-4d64-9195-c52539ff7c35-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f14c005c-83b1-4d64-9195-c52539ff7c35-openshift-service-ca\") pod \"f14c005c-83b1-4d64-9195-c52539ff7c35\" (UID: \"f14c005c-83b1-4d64-9195-c52539ff7c35\") "
Nov 03 16:45:54.677556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.677532    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f14c005c-83b1-4d64-9195-c52539ff7c35-openshift-service-ca" pod "f14c005c-83b1-4d64-9195-c52539ff7c35" (UID: "f14c005c-83b1-4d64-9195-c52539ff7c35"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:45:54.678020 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.677999    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f14c005c-83b1-4d64-9195-c52539ff7c35-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:54.679278 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.679253    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-maas-api-tls" pod "f14c005c-83b1-4d64-9195-c52539ff7c35" (UID: "f14c005c-83b1-4d64-9195-c52539ff7c35"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:45:54.679381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.679288    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f14c005c-83b1-4d64-9195-c52539ff7c35-kube-api-access-sjmkz" pod "f14c005c-83b1-4d64-9195-c52539ff7c35" (UID: "f14c005c-83b1-4d64-9195-c52539ff7c35"). InnerVolumeSpecName "kube-api-access-sjmkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:45:54.679381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.679322    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-metrics-certs" pod "f14c005c-83b1-4d64-9195-c52539ff7c35" (UID: "f14c005c-83b1-4d64-9195-c52539ff7c35"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:45:54.778613 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.778521    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-plugins-config-volume\") pod \"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7\" (UID: \"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7\") "
Nov 03 16:45:54.778837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.778682    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-kube-api-access-r5s7l\" (UniqueName: \"kubernetes.io/projected/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-kube-api-access-r5s7l\") pod \"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7\" (UID: \"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7\") "
Nov 03 16:45:54.778948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.778918    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-plugins-config-volume" pod "814d68eb-8cb5-4eea-8c2a-006a6e8b80e7" (UID: "814d68eb-8cb5-4eea-8c2a-006a6e8b80e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:45:54.779497 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.779476    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:54.779497 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.779497    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f14c005c-83b1-4d64-9195-c52539ff7c35-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:54.779650 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.779510    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:54.779650 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.779520    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjmkz\" (UniqueName: \"kubernetes.io/projected/f14c005c-83b1-4d64-9195-c52539ff7c35-kube-api-access-sjmkz\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:54.781136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.781111    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-kube-api-access-r5s7l" pod "814d68eb-8cb5-4eea-8c2a-006a6e8b80e7" (UID: "814d68eb-8cb5-4eea-8c2a-006a6e8b80e7"). InnerVolumeSpecName "kube-api-access-r5s7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:45:54.881054 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.881015    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5s7l\" (UniqueName: \"kubernetes.io/projected/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7-kube-api-access-r5s7l\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:45:54.881754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.881715    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"]
Nov 03 16:45:54.891326 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:54.891296    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-2537c1-6cf48f59c9-chcv9"]
Nov 03 16:45:55.568681 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:55.568581    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"
Nov 03 16:45:55.568681 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:55.568613    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2" event={"ID":"f14c005c-83b1-4d64-9195-c52539ff7c35","Type":"ContainerDied","Data":"4aa202b2dc6a47ee64d87fe600786fa133899b51612ef9e608e21ab991202284"}
Nov 03 16:45:55.569226 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:55.568717    5605 scope.go:122] "RemoveContainer" containerID="a0455e0331c2c9c94ff5de9b82706f034fdab0c86e10d29ac9e7ca17f222a893"
Nov 03 16:45:55.570185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:55.570159    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst" event={"ID":"814d68eb-8cb5-4eea-8c2a-006a6e8b80e7","Type":"ContainerDied","Data":"9fd2239c96dcd2b44484246071b5d4c90d141e3f2846918633a87fb213e83a22"}
Nov 03 16:45:55.570321 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:55.570213    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"
Nov 03 16:45:55.590278 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:55.580970    5605 scope.go:122] "RemoveContainer" containerID="af5f542bf6b808a100df23796a0fcd2a746b2e57a52233fca64abff48416562c"
Nov 03 16:45:55.748718 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:55.748681    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"]
Nov 03 16:45:55.799126 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:55.799092    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-2537c1-d85f5dcf9-qhtp2"]
Nov 03 16:45:55.869430 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:55.869393    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"]
Nov 03 16:45:55.873963 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:55.873937    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-2537c1-66bf8f76bc-dsqst"]
Nov 03 16:45:56.310580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:56.310510    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54734e4d-d0a1-4a3a-a617-fbffc691d3e5" path="/var/lib/kubelet/pods/54734e4d-d0a1-4a3a-a617-fbffc691d3e5/volumes"
Nov 03 16:45:56.311074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:56.311046    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="814d68eb-8cb5-4eea-8c2a-006a6e8b80e7" path="/var/lib/kubelet/pods/814d68eb-8cb5-4eea-8c2a-006a6e8b80e7/volumes"
Nov 03 16:45:56.311471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:45:56.311457    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f14c005c-83b1-4d64-9195-c52539ff7c35" path="/var/lib/kubelet/pods/f14c005c-83b1-4d64-9195-c52539ff7c35/volumes"
Nov 03 16:46:03.398222 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:46:03.398187    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:46:04.605650 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:04.605617    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"]
Nov 03 16:46:04.606695 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:04.605861    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy" containerID="cri-o://af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3" gracePeriod=30
Nov 03 16:46:09.895491 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.895423    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:46:09.911402 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911374    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-istio-token\" (UniqueName: \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-istio-token\") pod \"71409250-bb69-4586-8dc9-d22c888f1160\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") "
Nov 03 16:46:09.911541 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911442    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-envoy\") pod \"71409250-bb69-4586-8dc9-d22c888f1160\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") "
Nov 03 16:46:09.911541 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911464    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-data\") pod \"71409250-bb69-4586-8dc9-d22c888f1160\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") "
Nov 03 16:46:09.911541 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911520    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71409250-bb69-4586-8dc9-d22c888f1160-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71409250-bb69-4586-8dc9-d22c888f1160-istiod-ca-cert\") pod \"71409250-bb69-4586-8dc9-d22c888f1160\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") "
Nov 03 16:46:09.911715 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911548    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-socket\") pod \"71409250-bb69-4586-8dc9-d22c888f1160\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") "
Nov 03 16:46:09.911715 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911584    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-kube-api-access-rpxkr\" (UniqueName: \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-kube-api-access-rpxkr\") pod \"71409250-bb69-4586-8dc9-d22c888f1160\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") "
Nov 03 16:46:09.911715 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911635    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-certs\") pod \"71409250-bb69-4586-8dc9-d22c888f1160\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") "
Nov 03 16:46:09.911922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911719    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/71409250-bb69-4586-8dc9-d22c888f1160-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71409250-bb69-4586-8dc9-d22c888f1160-istio-podinfo\") pod \"71409250-bb69-4586-8dc9-d22c888f1160\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") "
Nov 03 16:46:09.911922 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911796    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-credential-socket\") pod \"71409250-bb69-4586-8dc9-d22c888f1160\" (UID: \"71409250-bb69-4586-8dc9-d22c888f1160\") "
Nov 03 16:46:09.912015 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911940    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-socket" pod "71409250-bb69-4586-8dc9-d22c888f1160" (UID: "71409250-bb69-4586-8dc9-d22c888f1160"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:09.912015 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.911972    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-certs" pod "71409250-bb69-4586-8dc9-d22c888f1160" (UID: "71409250-bb69-4586-8dc9-d22c888f1160"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:09.912107 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.912003    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-data" pod "71409250-bb69-4586-8dc9-d22c888f1160" (UID: "71409250-bb69-4586-8dc9-d22c888f1160"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:09.912214 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.912197    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71409250-bb69-4586-8dc9-d22c888f1160-istiod-ca-cert" pod "71409250-bb69-4586-8dc9-d22c888f1160" (UID: "71409250-bb69-4586-8dc9-d22c888f1160"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:46:09.912363 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.912337    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-credential-socket" pod "71409250-bb69-4586-8dc9-d22c888f1160" (UID: "71409250-bb69-4586-8dc9-d22c888f1160"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:09.912595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.912576    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:09.912658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.912603    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:09.912658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.912619    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:09.912658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.912634    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:09.912658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.912651    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71409250-bb69-4586-8dc9-d22c888f1160-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:09.913820 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.913796    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-istio-token" pod "71409250-bb69-4586-8dc9-d22c888f1160" (UID: "71409250-bb69-4586-8dc9-d22c888f1160"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:46:09.914127 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.914109    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-envoy" pod "71409250-bb69-4586-8dc9-d22c888f1160" (UID: "71409250-bb69-4586-8dc9-d22c888f1160"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:09.914823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.914799    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71409250-bb69-4586-8dc9-d22c888f1160-istio-podinfo" pod "71409250-bb69-4586-8dc9-d22c888f1160" (UID: "71409250-bb69-4586-8dc9-d22c888f1160"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:46:09.914823 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:09.914819    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-kube-api-access-rpxkr" pod "71409250-bb69-4586-8dc9-d22c888f1160" (UID: "71409250-bb69-4586-8dc9-d22c888f1160"). InnerVolumeSpecName "kube-api-access-rpxkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:46:10.014215 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.014176    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:10.014215 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.014209    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71409250-bb69-4586-8dc9-d22c888f1160-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:10.014215 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.014221    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpxkr\" (UniqueName: \"kubernetes.io/projected/71409250-bb69-4586-8dc9-d22c888f1160-kube-api-access-rpxkr\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:10.014447 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.014232    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71409250-bb69-4586-8dc9-d22c888f1160-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:10.711376 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.711332    5605 generic.go:356] "Generic (PLEG): container finished" podID="71409250-bb69-4586-8dc9-d22c888f1160" containerID="af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3" exitCode=0
Nov 03 16:46:10.711580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.711390    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" event={"ID":"71409250-bb69-4586-8dc9-d22c888f1160","Type":"ContainerDied","Data":"af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3"}
Nov 03 16:46:10.711580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.711464    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt" event={"ID":"71409250-bb69-4586-8dc9-d22c888f1160","Type":"ContainerDied","Data":"26069e90ef3c9e25ea913dac08a8c3c5e22a47da7c337d4c28b052a7ca41520b"}
Nov 03 16:46:10.711580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.711472    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"
Nov 03 16:46:10.711580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.711497    5605 scope.go:122] "RemoveContainer" containerID="af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3"
Nov 03 16:46:10.722456 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.722431    5605 scope.go:122] "RemoveContainer" containerID="eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b"
Nov 03 16:46:10.732635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.732404    5605 scope.go:122] "RemoveContainer" containerID="af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3"
Nov 03 16:46:10.732774 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:46:10.732752    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3\": container with ID starting with af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3 not found: ID does not exist" containerID="af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3"
Nov 03 16:46:10.732824 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.732783    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3"} err="failed to get container status \"af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3\": rpc error: code = NotFound desc = could not find container \"af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3\": container with ID starting with af8d88888c06a2d1ad6a502c449422e8e2c93d308a06c1cfcc798cbd79db00f3 not found: ID does not exist"
Nov 03 16:46:10.732824 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.732805    5605 scope.go:122] "RemoveContainer" containerID="eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b"
Nov 03 16:46:10.733082 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:46:10.733062    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b\": container with ID starting with eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b not found: ID does not exist" containerID="eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b"
Nov 03 16:46:10.733127 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.733088    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b"} err="failed to get container status \"eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b\": rpc error: code = NotFound desc = could not find container \"eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b\": container with ID starting with eb9bc642a2fa2b0834560571d1c934b0ddec7f4ba81c35b033a0d5b04b9d506b not found: ID does not exist"
Nov 03 16:46:10.897049 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.897017    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"]
Nov 03 16:46:10.947924 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:10.947889    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-2537c1-openshift-default-ff85d5c65-6v4pt"]
Nov 03 16:46:12.309544 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:12.309506    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71409250-bb69-4586-8dc9-d22c888f1160" path="/var/lib/kubelet/pods/71409250-bb69-4586-8dc9-d22c888f1160/volumes"
Nov 03 16:46:15.603368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.603333    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"]
Nov 03 16:46:15.604035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604018    5605 cpu_manager.go:414] "containerMap: removing container" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy"
Nov 03 16:46:15.604100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604037    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84b165c2-3d89-452c-852b-ffa8ca079c72" containerName="istio-proxy"
Nov 03 16:46:15.604100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604051    5605 cpu_manager.go:414] "containerMap: removing container" podUID="0b70f65a-9f12-458d-b942-61b7e178f019" containerName="cleanup"
Nov 03 16:46:15.604100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604056    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b70f65a-9f12-458d-b942-61b7e178f019" containerName="cleanup"
Nov 03 16:46:15.604100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604065    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f14c005c-83b1-4d64-9195-c52539ff7c35" containerName="maas-api"
Nov 03 16:46:15.604100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604070    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f14c005c-83b1-4d64-9195-c52539ff7c35" containerName="maas-api"
Nov 03 16:46:15.604100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604082    5605 cpu_manager.go:414] "containerMap: removing container" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy"
Nov 03 16:46:15.604100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604087    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy"
Nov 03 16:46:15.604100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604098    5605 cpu_manager.go:414] "containerMap: removing container" podUID="814d68eb-8cb5-4eea-8c2a-006a6e8b80e7" containerName="payload-pre-processing"
Nov 03 16:46:15.604100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604102    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="814d68eb-8cb5-4eea-8c2a-006a6e8b80e7" containerName="payload-pre-processing"
Nov 03 16:46:15.604374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604111    5605 cpu_manager.go:414] "containerMap: removing container" podUID="54734e4d-d0a1-4a3a-a617-fbffc691d3e5" containerName="payload-processing"
Nov 03 16:46:15.604374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604116    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54734e4d-d0a1-4a3a-a617-fbffc691d3e5" containerName="payload-processing"
Nov 03 16:46:15.604374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604134    5605 cpu_manager.go:414] "containerMap: removing container" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy"
Nov 03 16:46:15.604374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604139    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy"
Nov 03 16:46:15.604374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604233    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b70f65a-9f12-458d-b942-61b7e178f019" containerName="cleanup"
Nov 03 16:46:15.604374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604241    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f14c005c-83b1-4d64-9195-c52539ff7c35" containerName="maas-api"
Nov 03 16:46:15.604374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604250    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy"
Nov 03 16:46:15.604374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604257    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="814d68eb-8cb5-4eea-8c2a-006a6e8b80e7" containerName="payload-pre-processing"
Nov 03 16:46:15.604374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604264    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="54734e4d-d0a1-4a3a-a617-fbffc691d3e5" containerName="payload-processing"
Nov 03 16:46:15.604374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.604272    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="71409250-bb69-4586-8dc9-d22c888f1160" containerName="istio-proxy"
Nov 03 16:46:15.608668 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.608639    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:46:15.665918 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.665881    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"]
Nov 03 16:46:15.708759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.705129    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltzcb\" (UniqueName: \"kubernetes.io/projected/75eac520-9ac9-4564-a522-56cd310240c2-kube-api-access-ltzcb\") pod \"payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b\" (UID: \"75eac520-9ac9-4564-a522-56cd310240c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:46:15.708759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.705332    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75eac520-9ac9-4564-a522-56cd310240c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b\" (UID: \"75eac520-9ac9-4564-a522-56cd310240c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:46:15.733754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.727046    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-c626cd\""
Nov 03 16:46:15.733754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.727221    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-c626cd-dockercfg-l6zd2\""
Nov 03 16:46:15.820759 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.820708    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75eac520-9ac9-4564-a522-56cd310240c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b\" (UID: \"75eac520-9ac9-4564-a522-56cd310240c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:46:15.821210 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.821187    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltzcb\" (UniqueName: \"kubernetes.io/projected/75eac520-9ac9-4564-a522-56cd310240c2-kube-api-access-ltzcb\") pod \"payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b\" (UID: \"75eac520-9ac9-4564-a522-56cd310240c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:46:15.822080 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.821753    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75eac520-9ac9-4564-a522-56cd310240c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b\" (UID: \"75eac520-9ac9-4564-a522-56cd310240c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:46:15.883267 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.883195    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltzcb\" (UniqueName: \"kubernetes.io/projected/75eac520-9ac9-4564-a522-56cd310240c2-kube-api-access-ltzcb\") pod \"payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b\" (UID: \"75eac520-9ac9-4564-a522-56cd310240c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:46:15.932322 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:15.932288    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:46:16.331675 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:16.331648    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"]
Nov 03 16:46:16.342171 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:46:16.342126    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75eac520_9ac9_4564_a522_56cd310240c2.slice/crio-0afab97c805d6a0a112374941dae383e71b54bda69b8dc6ce268c4e36d377e12 WatchSource:0}: Error finding container 0afab97c805d6a0a112374941dae383e71b54bda69b8dc6ce268c4e36d377e12: Status 404 returned error can't find the container with id 0afab97c805d6a0a112374941dae383e71b54bda69b8dc6ce268c4e36d377e12
Nov 03 16:46:16.835743 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:16.835688    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b" event={"ID":"75eac520-9ac9-4564-a522-56cd310240c2","Type":"ContainerStarted","Data":"d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e"}
Nov 03 16:46:16.836154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:16.835773    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:46:16.836154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:16.835800    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b" event={"ID":"75eac520-9ac9-4564-a522-56cd310240c2","Type":"ContainerStarted","Data":"0afab97c805d6a0a112374941dae383e71b54bda69b8dc6ce268c4e36d377e12"}
Nov 03 16:46:16.957856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:16.957795    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b" podStartSLOduration=1.9577808060000002 podStartE2EDuration="1.957780806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:46:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:46:16.9563245 +0000 UTC m=+2062.780775770" watchObservedRunningTime="2026-11-03 16:46:16.957780806 +0000 UTC m=+2062.782232078"
Nov 03 16:46:24.048872 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:24.048834    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"]
Nov 03 16:46:24.164075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:24.164033    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"]
Nov 03 16:46:27.846471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:27.846439    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:46:29.050109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.050068    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4" podUID="c2e4bfb6-f892-4dae-8e86-36af9c6def94" containerName="payload-processing" containerID="cri-o://da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c" gracePeriod=25
Nov 03 16:46:29.165499 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.165401    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl" podUID="8ea72c89-5ec4-4822-be54-be0b1288d62c" containerName="payload-pre-processing" containerID="cri-o://caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8" gracePeriod=25
Nov 03 16:46:29.327621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.327566    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:46:29.429297 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.429263    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2e4bfb6-f892-4dae-8e86-36af9c6def94-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2e4bfb6-f892-4dae-8e86-36af9c6def94-plugins-config-volume\") pod \"c2e4bfb6-f892-4dae-8e86-36af9c6def94\" (UID: \"c2e4bfb6-f892-4dae-8e86-36af9c6def94\") "
Nov 03 16:46:29.429461 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.429328    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2e4bfb6-f892-4dae-8e86-36af9c6def94-kube-api-access-6pwff\" (UniqueName: \"kubernetes.io/projected/c2e4bfb6-f892-4dae-8e86-36af9c6def94-kube-api-access-6pwff\") pod \"c2e4bfb6-f892-4dae-8e86-36af9c6def94\" (UID: \"c2e4bfb6-f892-4dae-8e86-36af9c6def94\") "
Nov 03 16:46:29.429761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.429700    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2e4bfb6-f892-4dae-8e86-36af9c6def94-plugins-config-volume" pod "c2e4bfb6-f892-4dae-8e86-36af9c6def94" (UID: "c2e4bfb6-f892-4dae-8e86-36af9c6def94"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:46:29.430090 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.430072    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2e4bfb6-f892-4dae-8e86-36af9c6def94-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:29.431676 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.431653    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e4bfb6-f892-4dae-8e86-36af9c6def94-kube-api-access-6pwff" pod "c2e4bfb6-f892-4dae-8e86-36af9c6def94" (UID: "c2e4bfb6-f892-4dae-8e86-36af9c6def94"). InnerVolumeSpecName "kube-api-access-6pwff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:46:29.453101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.453073    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:46:29.531360 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.531321    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ea72c89-5ec4-4822-be54-be0b1288d62c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea72c89-5ec4-4822-be54-be0b1288d62c-plugins-config-volume\") pod \"8ea72c89-5ec4-4822-be54-be0b1288d62c\" (UID: \"8ea72c89-5ec4-4822-be54-be0b1288d62c\") "
Nov 03 16:46:29.531557 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.531378    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ea72c89-5ec4-4822-be54-be0b1288d62c-kube-api-access-2z7d6\" (UniqueName: \"kubernetes.io/projected/8ea72c89-5ec4-4822-be54-be0b1288d62c-kube-api-access-2z7d6\") pod \"8ea72c89-5ec4-4822-be54-be0b1288d62c\" (UID: \"8ea72c89-5ec4-4822-be54-be0b1288d62c\") "
Nov 03 16:46:29.531780 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.531744    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ea72c89-5ec4-4822-be54-be0b1288d62c-plugins-config-volume" pod "8ea72c89-5ec4-4822-be54-be0b1288d62c" (UID: "8ea72c89-5ec4-4822-be54-be0b1288d62c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:46:29.532035 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.532017    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ea72c89-5ec4-4822-be54-be0b1288d62c-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:29.532075 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.532038    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pwff\" (UniqueName: \"kubernetes.io/projected/c2e4bfb6-f892-4dae-8e86-36af9c6def94-kube-api-access-6pwff\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:29.533767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.533745    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ea72c89-5ec4-4822-be54-be0b1288d62c-kube-api-access-2z7d6" pod "8ea72c89-5ec4-4822-be54-be0b1288d62c" (UID: "8ea72c89-5ec4-4822-be54-be0b1288d62c"). InnerVolumeSpecName "kube-api-access-2z7d6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:46:29.633646 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.633610    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z7d6\" (UniqueName: \"kubernetes.io/projected/8ea72c89-5ec4-4822-be54-be0b1288d62c-kube-api-access-2z7d6\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:29.921696 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.921600    5605 generic.go:356] "Generic (PLEG): container finished" podID="8ea72c89-5ec4-4822-be54-be0b1288d62c" containerID="caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8" exitCode=0
Nov 03 16:46:29.921696 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.921645    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"
Nov 03 16:46:29.921955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.921688    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl" event={"ID":"8ea72c89-5ec4-4822-be54-be0b1288d62c","Type":"ContainerDied","Data":"caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8"}
Nov 03 16:46:29.921955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.921774    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl" event={"ID":"8ea72c89-5ec4-4822-be54-be0b1288d62c","Type":"ContainerDied","Data":"d377dd7cd62fc77485a113deb1de9f7e654e33443ee90fafb5810b164ebe3b6d"}
Nov 03 16:46:29.921955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.921798    5605 scope.go:122] "RemoveContainer" containerID="caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8"
Nov 03 16:46:29.923227 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.923203    5605 generic.go:356] "Generic (PLEG): container finished" podID="c2e4bfb6-f892-4dae-8e86-36af9c6def94" containerID="da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c" exitCode=0
Nov 03 16:46:29.923352 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.923249    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"
Nov 03 16:46:29.923352 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.923286    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4" event={"ID":"c2e4bfb6-f892-4dae-8e86-36af9c6def94","Type":"ContainerDied","Data":"da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c"}
Nov 03 16:46:29.923352 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.923332    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4" event={"ID":"c2e4bfb6-f892-4dae-8e86-36af9c6def94","Type":"ContainerDied","Data":"a285f2409f31d5cbb2f6250b336f7665851e0cf39f2692d7b1153de3e8dd21a9"}
Nov 03 16:46:29.932253 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.932238    5605 scope.go:122] "RemoveContainer" containerID="caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8"
Nov 03 16:46:29.932550 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:46:29.932530    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8\": container with ID starting with caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8 not found: ID does not exist" containerID="caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8"
Nov 03 16:46:29.932657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.932554    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8"} err="failed to get container status \"caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8\": rpc error: code = NotFound desc = could not find container \"caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8\": container with ID starting with caa22eb2a50b1145e6afcb807767005a7b22d7d6c77532cf56d7ccbd5bb9e9a8 not found: ID does not exist"
Nov 03 16:46:29.932657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.932571    5605 scope.go:122] "RemoveContainer" containerID="da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c"
Nov 03 16:46:29.942162 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.942146    5605 scope.go:122] "RemoveContainer" containerID="da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c"
Nov 03 16:46:29.942449 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:46:29.942429    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c\": container with ID starting with da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c not found: ID does not exist" containerID="da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c"
Nov 03 16:46:29.942520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:29.942456    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c"} err="failed to get container status \"da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c\": rpc error: code = NotFound desc = could not find container \"da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c\": container with ID starting with da2e8ee41534f321ab02c9c378856869efc40b1d4f2011abde708d9bbaadbb0c not found: ID does not exist"
Nov 03 16:46:30.103184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:30.103150    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"]
Nov 03 16:46:30.153865 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:30.153834    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cb494a-6fb67f5895-5llgl"]
Nov 03 16:46:30.222790 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:30.222659    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"]
Nov 03 16:46:30.229937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:30.229911    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cb494a-74dd899c4f-j4vq4"]
Nov 03 16:46:30.309984 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:30.309945    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ea72c89-5ec4-4822-be54-be0b1288d62c" path="/var/lib/kubelet/pods/8ea72c89-5ec4-4822-be54-be0b1288d62c/volumes"
Nov 03 16:46:30.310526 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:30.310504    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2e4bfb6-f892-4dae-8e86-36af9c6def94" path="/var/lib/kubelet/pods/c2e4bfb6-f892-4dae-8e86-36af9c6def94/volumes"
Nov 03 16:46:33.393240 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:46:33.393200    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:46:42.431231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:42.431199    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"]
Nov 03 16:46:42.431693 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:42.431460    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy" containerID="cri-o://3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e" gracePeriod=30
Nov 03 16:46:42.511294 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:42.511252    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"]
Nov 03 16:46:42.511549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:42.511518    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy" containerID="cri-o://086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc" gracePeriod=30
Nov 03 16:46:42.514120 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:42.514094    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"]
Nov 03 16:46:42.514330 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:42.514304    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy" containerID="cri-o://0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4" gracePeriod=30
Nov 03 16:46:47.059657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:47.059576    5605 generic.go:356] "Generic (PLEG): container finished" podID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerID="73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531" exitCode=0
Nov 03 16:46:47.059657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:47.059646    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" event={"ID":"8114cb0a-6d4e-4b87-b562-03cf2fec6319","Type":"ContainerDied","Data":"73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531"}
Nov 03 16:46:47.060213 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:47.060193    5605 scope.go:122] "RemoveContainer" containerID="73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531"
Nov 03 16:46:47.061974 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:47.061943    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:46:47.062041 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:47.062016    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:46:47.062083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:47.062049    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:46:47.797860 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:47.797826    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:46:47.840559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:47.840534    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:46:47.876331 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:47.876309    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:46:48.006267 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006165    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-certs\") pod \"03b5a7fa-c4c4-4778-a652-09969f86569e\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") "
Nov 03 16:46:48.006458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006276    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-socket\") pod \"e6e016ec-3eaf-4d13-a322-425d83281dc0\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") "
Nov 03 16:46:48.006458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006321    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-kube-api-access-gcd5x\" (UniqueName: \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-kube-api-access-gcd5x\") pod \"03b5a7fa-c4c4-4778-a652-09969f86569e\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") "
Nov 03 16:46:48.006458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006356    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-data\") pod \"266192f8-6a4e-4736-b400-22433bd0e632\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") "
Nov 03 16:46:48.006637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006459    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-credential-socket\") pod \"266192f8-6a4e-4736-b400-22433bd0e632\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") "
Nov 03 16:46:48.006637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006510    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-data\") pod \"e6e016ec-3eaf-4d13-a322-425d83281dc0\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") "
Nov 03 16:46:48.006637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006517    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-certs" pod "03b5a7fa-c4c4-4778-a652-09969f86569e" (UID: "03b5a7fa-c4c4-4778-a652-09969f86569e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.006637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006541    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-kube-api-access-jxpgb\" (UniqueName: \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-kube-api-access-jxpgb\") pod \"e6e016ec-3eaf-4d13-a322-425d83281dc0\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") "
Nov 03 16:46:48.006637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006601    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-socket" pod "e6e016ec-3eaf-4d13-a322-425d83281dc0" (UID: "e6e016ec-3eaf-4d13-a322-425d83281dc0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.006637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006607    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-data" pod "266192f8-6a4e-4736-b400-22433bd0e632" (UID: "266192f8-6a4e-4736-b400-22433bd0e632"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.006637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006623    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03b5a7fa-c4c4-4778-a652-09969f86569e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03b5a7fa-c4c4-4778-a652-09969f86569e-istiod-ca-cert\") pod \"03b5a7fa-c4c4-4778-a652-09969f86569e\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") "
Nov 03 16:46:48.007069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006676    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-envoy\") pod \"e6e016ec-3eaf-4d13-a322-425d83281dc0\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") "
Nov 03 16:46:48.007069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006777    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-data\") pod \"03b5a7fa-c4c4-4778-a652-09969f86569e\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") "
Nov 03 16:46:48.007069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006810    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-token\" (UniqueName: \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-token\") pod \"e6e016ec-3eaf-4d13-a322-425d83281dc0\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") "
Nov 03 16:46:48.007069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006841    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/266192f8-6a4e-4736-b400-22433bd0e632-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/266192f8-6a4e-4736-b400-22433bd0e632-istiod-ca-cert\") pod \"266192f8-6a4e-4736-b400-22433bd0e632\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") "
Nov 03 16:46:48.007069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006866    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-credential-socket" pod "266192f8-6a4e-4736-b400-22433bd0e632" (UID: "266192f8-6a4e-4736-b400-22433bd0e632"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.007069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006873    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-istio-token\" (UniqueName: \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-istio-token\") pod \"266192f8-6a4e-4736-b400-22433bd0e632\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") "
Nov 03 16:46:48.007069 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006942    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-podinfo\") pod \"e6e016ec-3eaf-4d13-a322-425d83281dc0\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") "
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007087    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-socket\") pod \"266192f8-6a4e-4736-b400-22433bd0e632\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") "
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006884    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-data" pod "e6e016ec-3eaf-4d13-a322-425d83281dc0" (UID: "e6e016ec-3eaf-4d13-a322-425d83281dc0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.006999    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-data" pod "03b5a7fa-c4c4-4778-a652-09969f86569e" (UID: "03b5a7fa-c4c4-4778-a652-09969f86569e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007091    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03b5a7fa-c4c4-4778-a652-09969f86569e-istiod-ca-cert" pod "03b5a7fa-c4c4-4778-a652-09969f86569e" (UID: "03b5a7fa-c4c4-4778-a652-09969f86569e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007131    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-certs\") pod \"e6e016ec-3eaf-4d13-a322-425d83281dc0\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") "
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007160    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-credential-socket\") pod \"e6e016ec-3eaf-4d13-a322-425d83281dc0\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") "
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007191    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-socket\") pod \"03b5a7fa-c4c4-4778-a652-09969f86569e\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") "
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007229    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-envoy\") pod \"03b5a7fa-c4c4-4778-a652-09969f86569e\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") "
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007294    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/266192f8-6a4e-4736-b400-22433bd0e632-istiod-ca-cert" pod "266192f8-6a4e-4736-b400-22433bd0e632" (UID: "266192f8-6a4e-4736-b400-22433bd0e632"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007300    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-token\" (UniqueName: \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-token\") pod \"03b5a7fa-c4c4-4778-a652-09969f86569e\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") "
Nov 03 16:46:48.007439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007359    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-envoy\") pod \"266192f8-6a4e-4736-b400-22433bd0e632\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") "
Nov 03 16:46:48.008209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007578    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-certs\") pod \"266192f8-6a4e-4736-b400-22433bd0e632\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") "
Nov 03 16:46:48.008209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007612    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-podinfo\") pod \"03b5a7fa-c4c4-4778-a652-09969f86569e\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") "
Nov 03 16:46:48.008209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007646    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/266192f8-6a4e-4736-b400-22433bd0e632-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/266192f8-6a4e-4736-b400-22433bd0e632-istio-podinfo\") pod \"266192f8-6a4e-4736-b400-22433bd0e632\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") "
Nov 03 16:46:48.008209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007677    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-kube-api-access-6cfd5\" (UniqueName: \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-kube-api-access-6cfd5\") pod \"266192f8-6a4e-4736-b400-22433bd0e632\" (UID: \"266192f8-6a4e-4736-b400-22433bd0e632\") "
Nov 03 16:46:48.008209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007767    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-credential-socket\") pod \"03b5a7fa-c4c4-4778-a652-09969f86569e\" (UID: \"03b5a7fa-c4c4-4778-a652-09969f86569e\") "
Nov 03 16:46:48.008209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.007825    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6e016ec-3eaf-4d13-a322-425d83281dc0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6e016ec-3eaf-4d13-a322-425d83281dc0-istiod-ca-cert\") pod \"e6e016ec-3eaf-4d13-a322-425d83281dc0\" (UID: \"e6e016ec-3eaf-4d13-a322-425d83281dc0\") "
Nov 03 16:46:48.008552 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008513    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-socket" pod "266192f8-6a4e-4736-b400-22433bd0e632" (UID: "266192f8-6a4e-4736-b400-22433bd0e632"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008616    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008640    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008657    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008672    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008686    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008700    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03b5a7fa-c4c4-4778-a652-09969f86569e-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008713    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008745    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/266192f8-6a4e-4736-b400-22433bd0e632-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008744    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-socket" pod "03b5a7fa-c4c4-4778-a652-09969f86569e" (UID: "03b5a7fa-c4c4-4778-a652-09969f86569e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008748    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-certs" pod "266192f8-6a4e-4736-b400-22433bd0e632" (UID: "266192f8-6a4e-4736-b400-22433bd0e632"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008743    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-credential-socket" pod "e6e016ec-3eaf-4d13-a322-425d83281dc0" (UID: "e6e016ec-3eaf-4d13-a322-425d83281dc0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.009065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.008911    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-credential-socket" pod "03b5a7fa-c4c4-4778-a652-09969f86569e" (UID: "03b5a7fa-c4c4-4778-a652-09969f86569e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.009890 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.009072    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6e016ec-3eaf-4d13-a322-425d83281dc0-istiod-ca-cert" pod "e6e016ec-3eaf-4d13-a322-425d83281dc0" (UID: "e6e016ec-3eaf-4d13-a322-425d83281dc0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:46:48.009890 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.009245    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-certs" pod "e6e016ec-3eaf-4d13-a322-425d83281dc0" (UID: "e6e016ec-3eaf-4d13-a322-425d83281dc0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.010492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.010443    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-istio-token" pod "266192f8-6a4e-4736-b400-22433bd0e632" (UID: "266192f8-6a4e-4736-b400-22433bd0e632"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:46:48.010608 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.010540    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-kube-api-access-jxpgb" pod "e6e016ec-3eaf-4d13-a322-425d83281dc0" (UID: "e6e016ec-3eaf-4d13-a322-425d83281dc0"). InnerVolumeSpecName "kube-api-access-jxpgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:46:48.010916 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.010889    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-token" pod "e6e016ec-3eaf-4d13-a322-425d83281dc0" (UID: "e6e016ec-3eaf-4d13-a322-425d83281dc0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:46:48.011297 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.010990    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-kube-api-access-gcd5x" pod "03b5a7fa-c4c4-4778-a652-09969f86569e" (UID: "03b5a7fa-c4c4-4778-a652-09969f86569e"). InnerVolumeSpecName "kube-api-access-gcd5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:46:48.011375 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.011290    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-envoy" pod "e6e016ec-3eaf-4d13-a322-425d83281dc0" (UID: "e6e016ec-3eaf-4d13-a322-425d83281dc0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.011647 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.011619    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-token" pod "03b5a7fa-c4c4-4778-a652-09969f86569e" (UID: "03b5a7fa-c4c4-4778-a652-09969f86569e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:46:48.011777 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.011710    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-podinfo" pod "e6e016ec-3eaf-4d13-a322-425d83281dc0" (UID: "e6e016ec-3eaf-4d13-a322-425d83281dc0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:46:48.011845 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.011769    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/266192f8-6a4e-4736-b400-22433bd0e632-istio-podinfo" pod "266192f8-6a4e-4736-b400-22433bd0e632" (UID: "266192f8-6a4e-4736-b400-22433bd0e632"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:46:48.011990 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.011970    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-kube-api-access-6cfd5" pod "266192f8-6a4e-4736-b400-22433bd0e632" (UID: "266192f8-6a4e-4736-b400-22433bd0e632"). InnerVolumeSpecName "kube-api-access-6cfd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:46:48.012237 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.012220    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-envoy" pod "03b5a7fa-c4c4-4778-a652-09969f86569e" (UID: "03b5a7fa-c4c4-4778-a652-09969f86569e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.012324 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.012310    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-envoy" pod "266192f8-6a4e-4736-b400-22433bd0e632" (UID: "266192f8-6a4e-4736-b400-22433bd0e632"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:46:48.013204 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.013184    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-podinfo" pod "03b5a7fa-c4c4-4778-a652-09969f86569e" (UID: "03b5a7fa-c4c4-4778-a652-09969f86569e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:46:48.067427 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.067390    5605 generic.go:356] "Generic (PLEG): container finished" podID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerID="3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e" exitCode=0
Nov 03 16:46:48.067884 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.067444    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"
Nov 03 16:46:48.067884 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.067469    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" event={"ID":"03b5a7fa-c4c4-4778-a652-09969f86569e","Type":"ContainerDied","Data":"3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e"}
Nov 03 16:46:48.067884 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.067521    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg" event={"ID":"03b5a7fa-c4c4-4778-a652-09969f86569e","Type":"ContainerDied","Data":"ef7194693f5746b874aff4893c5623b502587aca92a922a5e2e96bd65f1f66b8"}
Nov 03 16:46:48.067884 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.067548    5605 scope.go:122] "RemoveContainer" containerID="3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e"
Nov 03 16:46:48.069947 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.069914    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" event={"ID":"8114cb0a-6d4e-4b87-b562-03cf2fec6319","Type":"ContainerStarted","Data":"049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085"}
Nov 03 16:46:48.071426 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.071403    5605 generic.go:356] "Generic (PLEG): container finished" podID="266192f8-6a4e-4736-b400-22433bd0e632" containerID="086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc" exitCode=0
Nov 03 16:46:48.071551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.071450    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"
Nov 03 16:46:48.071551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.071481    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" event={"ID":"266192f8-6a4e-4736-b400-22433bd0e632","Type":"ContainerDied","Data":"086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc"}
Nov 03 16:46:48.071551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.071516    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk" event={"ID":"266192f8-6a4e-4736-b400-22433bd0e632","Type":"ContainerDied","Data":"78908e3abc2edfeb173678ce6c0f9b691b91d6b7e0a1e5320bada9bc48370d90"}
Nov 03 16:46:48.074188 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.074163    5605 generic.go:356] "Generic (PLEG): container finished" podID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerID="0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4" exitCode=0
Nov 03 16:46:48.074348 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.074205    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"
Nov 03 16:46:48.075129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.074806    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" event={"ID":"e6e016ec-3eaf-4d13-a322-425d83281dc0","Type":"ContainerDied","Data":"0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4"}
Nov 03 16:46:48.075129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.074856    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6" event={"ID":"e6e016ec-3eaf-4d13-a322-425d83281dc0","Type":"ContainerDied","Data":"59b53b0a9b09b4bd67aa3fbd68c76f56fe389077778ed8be2b9677bad6c60df5"}
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.108908    5605 scope.go:122] "RemoveContainer" containerID="3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e"
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110376    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110406    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110426    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110440    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110455    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110468    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110483    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110497    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/266192f8-6a4e-4736-b400-22433bd0e632-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110511    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03b5a7fa-c4c4-4778-a652-09969f86569e-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110524    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/266192f8-6a4e-4736-b400-22433bd0e632-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110539    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cfd5\" (UniqueName: \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-kube-api-access-6cfd5\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110554    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03b5a7fa-c4c4-4778-a652-09969f86569e-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110570    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6e016ec-3eaf-4d13-a322-425d83281dc0-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110585    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcd5x\" (UniqueName: \"kubernetes.io/projected/03b5a7fa-c4c4-4778-a652-09969f86569e-kube-api-access-gcd5x\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110599    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxpgb\" (UniqueName: \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-kube-api-access-jxpgb\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110613    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110626    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110641    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/266192f8-6a4e-4736-b400-22433bd0e632-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.110655    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6e016ec-3eaf-4d13-a322-425d83281dc0-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:46:48.113274    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e\": container with ID starting with 3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e not found: ID does not exist" containerID="3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e"
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.113317    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e"} err="failed to get container status \"3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e\": rpc error: code = NotFound desc = could not find container \"3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e\": container with ID starting with 3d743ddc7d8ea3ec0f9dd4736ec249c6ebcb312842872446369f12669f8c094e not found: ID does not exist"
Nov 03 16:46:48.118009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.113345    5605 scope.go:122] "RemoveContainer" containerID="086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc"
Nov 03 16:46:48.133153 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.133100    5605 scope.go:122] "RemoveContainer" containerID="086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc"
Nov 03 16:46:48.133706 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:46:48.133679    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc\": container with ID starting with 086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc not found: ID does not exist" containerID="086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc"
Nov 03 16:46:48.133836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.133719    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc"} err="failed to get container status \"086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc\": rpc error: code = NotFound desc = could not find container \"086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc\": container with ID starting with 086bba1f6a9b8ed967f198486c4dc05bc7d0f0f44930931739d5a469cb34fcfc not found: ID does not exist"
Nov 03 16:46:48.133836 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.133783    5605 scope.go:122] "RemoveContainer" containerID="0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4"
Nov 03 16:46:48.144027 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.143983    5605 scope.go:122] "RemoveContainer" containerID="0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4"
Nov 03 16:46:48.144314 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:46:48.144282    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4\": container with ID starting with 0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4 not found: ID does not exist" containerID="0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4"
Nov 03 16:46:48.144382 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.144327    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4"} err="failed to get container status \"0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4\": rpc error: code = NotFound desc = could not find container \"0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4\": container with ID starting with 0799b747ac38ae7743aec299268ce030f7ddd7a7a0f367cf3b6a8489ad3bc6a4 not found: ID does not exist"
Nov 03 16:46:48.248624 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.248593    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:46:48.249511 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.249481    5605 patch_prober.go:29] interesting pod/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Nov 03 16:46:48.249649 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.249540    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Nov 03 16:46:48.475661 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.475624    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"]
Nov 03 16:46:48.507166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.507109    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-kklqk"]
Nov 03 16:46:48.610807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.608403    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"]
Nov 03 16:46:48.613492 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.613465    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vlxdg"]
Nov 03 16:46:48.730394 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.730300    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"]
Nov 03 16:46:48.735929 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.735900    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-trzd6"]
Nov 03 16:46:49.000816 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:48.995842    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:46:49.249357 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:49.249324    5605 patch_prober.go:29] interesting pod/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Nov 03 16:46:49.249769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:49.249383    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Nov 03 16:46:50.248948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:50.248911    5605 patch_prober.go:29] interesting pod/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Nov 03 16:46:50.249111 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:50.248971    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Nov 03 16:46:50.322391 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:50.322352    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" path="/var/lib/kubelet/pods/03b5a7fa-c4c4-4778-a652-09969f86569e/volumes"
Nov 03 16:46:50.324418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:50.324388    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="266192f8-6a4e-4736-b400-22433bd0e632" path="/var/lib/kubelet/pods/266192f8-6a4e-4736-b400-22433bd0e632/volumes"
Nov 03 16:46:50.325076 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:50.325057    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" path="/var/lib/kubelet/pods/e6e016ec-3eaf-4d13-a322-425d83281dc0/volumes"
Nov 03 16:46:51.249396 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:51.249359    5605 patch_prober.go:29] interesting pod/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Nov 03 16:46:51.249575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:51.249423    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Nov 03 16:46:52.234326 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:52.234290    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"]
Nov 03 16:46:52.248811 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:52.248780    5605 patch_prober.go:29] interesting pod/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Nov 03 16:46:52.248935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:52.248837    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Nov 03 16:46:53.248798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:53.248767    5605 patch_prober.go:29] interesting pod/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Nov 03 16:46:53.249200 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:53.248818    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Nov 03 16:46:54.254208 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:54.254181    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:46:54.255482 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:54.255461    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:46:54.605356 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:54.605326    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:46:54.619881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:46:54.619851    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:47:03.392990 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:03.392897    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:47:07.256561 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.256521    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv" podUID="9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" containerName="main" containerID="cri-o://1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac" gracePeriod=45
Nov 03 16:47:07.595711 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.595685    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:47:07.703181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.703146    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tmp-dir\") pod \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") "
Nov 03 16:47:07.703378 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.703208    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tls-certs\") pod \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") "
Nov 03 16:47:07.703378 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.703254    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-model-cache\") pod \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") "
Nov 03 16:47:07.703378 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.703282    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-home\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-home\") pod \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") "
Nov 03 16:47:07.703378 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.703360    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-kube-api-access-7qwd7\" (UniqueName: \"kubernetes.io/projected/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-kube-api-access-7qwd7\") pod \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") "
Nov 03 16:47:07.703583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.703426    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-dshm\") pod \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\" (UID: \"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f\") "
Nov 03 16:47:07.703583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.703445    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tmp-dir" pod "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" (UID: "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:07.703583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.703482    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-model-cache" pod "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" (UID: "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:07.703775 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.703592    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-home" pod "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" (UID: "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:07.704043 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.704023    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:07.704142 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.704065    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:07.704142 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.704094    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:07.705885 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.705854    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tls-certs" pod "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" (UID: "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:47:07.706012 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.705896    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-kube-api-access-7qwd7" pod "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" (UID: "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f"). InnerVolumeSpecName "kube-api-access-7qwd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:47:07.707852 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.707828    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-dshm" pod "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" (UID: "9cd488c1-cb1d-42e0-bec6-be52b44f1c1f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:07.805096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.805005    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:07.805096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.805039    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qwd7\" (UniqueName: \"kubernetes.io/projected/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-kube-api-access-7qwd7\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:07.805096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:07.805055    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:08.284470 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:08.284433    5605 generic.go:356] "Generic (PLEG): container finished" podID="9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" containerID="1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac" exitCode=0
Nov 03 16:47:08.284969 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:08.284514    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"
Nov 03 16:47:08.284969 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:08.284512    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv" event={"ID":"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f","Type":"ContainerDied","Data":"1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac"}
Nov 03 16:47:08.284969 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:08.284648    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv" event={"ID":"9cd488c1-cb1d-42e0-bec6-be52b44f1c1f","Type":"ContainerDied","Data":"b13afb1ab55204345920cb081892841d3f0faaaf451b9959528f71a0167fce21"}
Nov 03 16:47:08.284969 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:08.284673    5605 scope.go:122] "RemoveContainer" containerID="1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac"
Nov 03 16:47:08.295859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:08.295842    5605 scope.go:122] "RemoveContainer" containerID="1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac"
Nov 03 16:47:08.296129 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:08.296112    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac\": container with ID starting with 1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac not found: ID does not exist" containerID="1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac"
Nov 03 16:47:08.296175 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:08.296136    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac"} err="failed to get container status \"1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac\": rpc error: code = NotFound desc = could not find container \"1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac\": container with ID starting with 1795e15158cd0c3348cbe8a7a9c5172d984f8309c7c7f88ecea0040a2200bbac not found: ID does not exist"
Nov 03 16:47:08.453548 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:08.453504    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv" podUID="9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:47:08.459122 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:08.459089    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"]
Nov 03 16:47:08.510525 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:08.510494    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-9635db0d/e2e-lifecycle-model-9635db0d-kserve-b6868b89c-z4zsv"]
Nov 03 16:47:09.283360 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:09.283328    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"]
Nov 03 16:47:09.398791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:09.398756    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"]
Nov 03 16:47:10.309370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:10.309336    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" path="/var/lib/kubelet/pods/9cd488c1-cb1d-42e0-bec6-be52b44f1c1f/volumes"
Nov 03 16:47:14.284157 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.284081    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f" podUID="eade339c-4ff4-49d3-b634-1ad4459904f3" containerName="payload-processing" containerID="cri-o://3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5" gracePeriod=25
Nov 03 16:47:14.399779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.399738    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt" podUID="978db8fe-4aa9-40a6-8183-def3b49e2b68" containerName="payload-pre-processing" containerID="cri-o://7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373" gracePeriod=25
Nov 03 16:47:14.558288 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.558259    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:47:14.645657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.645625    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eade339c-4ff4-49d3-b634-1ad4459904f3-kube-api-access-2tb9s\" (UniqueName: \"kubernetes.io/projected/eade339c-4ff4-49d3-b634-1ad4459904f3-kube-api-access-2tb9s\") pod \"eade339c-4ff4-49d3-b634-1ad4459904f3\" (UID: \"eade339c-4ff4-49d3-b634-1ad4459904f3\") "
Nov 03 16:47:14.645872 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.645774    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eade339c-4ff4-49d3-b634-1ad4459904f3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eade339c-4ff4-49d3-b634-1ad4459904f3-plugins-config-volume\") pod \"eade339c-4ff4-49d3-b634-1ad4459904f3\" (UID: \"eade339c-4ff4-49d3-b634-1ad4459904f3\") "
Nov 03 16:47:14.646273 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.646247    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eade339c-4ff4-49d3-b634-1ad4459904f3-plugins-config-volume" pod "eade339c-4ff4-49d3-b634-1ad4459904f3" (UID: "eade339c-4ff4-49d3-b634-1ad4459904f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:47:14.648349 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.648321    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eade339c-4ff4-49d3-b634-1ad4459904f3-kube-api-access-2tb9s" pod "eade339c-4ff4-49d3-b634-1ad4459904f3" (UID: "eade339c-4ff4-49d3-b634-1ad4459904f3"). InnerVolumeSpecName "kube-api-access-2tb9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:47:14.669915 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.669892    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:47:14.747138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.747094    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/978db8fe-4aa9-40a6-8183-def3b49e2b68-kube-api-access-7dp9m\" (UniqueName: \"kubernetes.io/projected/978db8fe-4aa9-40a6-8183-def3b49e2b68-kube-api-access-7dp9m\") pod \"978db8fe-4aa9-40a6-8183-def3b49e2b68\" (UID: \"978db8fe-4aa9-40a6-8183-def3b49e2b68\") "
Nov 03 16:47:14.747310 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.747160    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/978db8fe-4aa9-40a6-8183-def3b49e2b68-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/978db8fe-4aa9-40a6-8183-def3b49e2b68-plugins-config-volume\") pod \"978db8fe-4aa9-40a6-8183-def3b49e2b68\" (UID: \"978db8fe-4aa9-40a6-8183-def3b49e2b68\") "
Nov 03 16:47:14.747523 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.747506    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tb9s\" (UniqueName: \"kubernetes.io/projected/eade339c-4ff4-49d3-b634-1ad4459904f3-kube-api-access-2tb9s\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:14.747599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.747526    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eade339c-4ff4-49d3-b634-1ad4459904f3-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:14.747599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.747506    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/978db8fe-4aa9-40a6-8183-def3b49e2b68-plugins-config-volume" pod "978db8fe-4aa9-40a6-8183-def3b49e2b68" (UID: "978db8fe-4aa9-40a6-8183-def3b49e2b68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:47:14.749339 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.749320    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/978db8fe-4aa9-40a6-8183-def3b49e2b68-kube-api-access-7dp9m" pod "978db8fe-4aa9-40a6-8183-def3b49e2b68" (UID: "978db8fe-4aa9-40a6-8183-def3b49e2b68"). InnerVolumeSpecName "kube-api-access-7dp9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:47:14.849091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.849066    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dp9m\" (UniqueName: \"kubernetes.io/projected/978db8fe-4aa9-40a6-8183-def3b49e2b68-kube-api-access-7dp9m\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:14.849091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:14.849092    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/978db8fe-4aa9-40a6-8183-def3b49e2b68-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:15.340112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.340078    5605 generic.go:356] "Generic (PLEG): container finished" podID="eade339c-4ff4-49d3-b634-1ad4459904f3" containerID="3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5" exitCode=0
Nov 03 16:47:15.340576 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.340126    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"
Nov 03 16:47:15.340576 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.340171    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f" event={"ID":"eade339c-4ff4-49d3-b634-1ad4459904f3","Type":"ContainerDied","Data":"3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5"}
Nov 03 16:47:15.340576 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.340226    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f" event={"ID":"eade339c-4ff4-49d3-b634-1ad4459904f3","Type":"ContainerDied","Data":"7f937d05323efdad7076878b981ed88ce18727aa87dc0b4de873967a9f5a8921"}
Nov 03 16:47:15.340576 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.340269    5605 scope.go:122] "RemoveContainer" containerID="3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5"
Nov 03 16:47:15.342237 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.342212    5605 generic.go:356] "Generic (PLEG): container finished" podID="978db8fe-4aa9-40a6-8183-def3b49e2b68" containerID="7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373" exitCode=0
Nov 03 16:47:15.342375 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.342268    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt" event={"ID":"978db8fe-4aa9-40a6-8183-def3b49e2b68","Type":"ContainerDied","Data":"7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373"}
Nov 03 16:47:15.342375 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.342272    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"
Nov 03 16:47:15.342375 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.342307    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt" event={"ID":"978db8fe-4aa9-40a6-8183-def3b49e2b68","Type":"ContainerDied","Data":"fc25b3f23e098764b57900eba3f6af429c9ee7968dd059506fe884ae38f8b179"}
Nov 03 16:47:15.352696 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.352675    5605 scope.go:122] "RemoveContainer" containerID="3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5"
Nov 03 16:47:15.353054 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:15.353021    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5\": container with ID starting with 3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5 not found: ID does not exist" containerID="3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5"
Nov 03 16:47:15.353144 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.353069    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5"} err="failed to get container status \"3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5\": rpc error: code = NotFound desc = could not find container \"3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5\": container with ID starting with 3e88a6a67042948f47d9b7040edb49a694902d41e1538179f65863a3cb7fc3e5 not found: ID does not exist"
Nov 03 16:47:15.353144 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.353091    5605 scope.go:122] "RemoveContainer" containerID="7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373"
Nov 03 16:47:15.362513 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.362497    5605 scope.go:122] "RemoveContainer" containerID="7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373"
Nov 03 16:47:15.362774 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:15.362753    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373\": container with ID starting with 7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373 not found: ID does not exist" containerID="7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373"
Nov 03 16:47:15.362851 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.362779    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373"} err="failed to get container status \"7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373\": rpc error: code = NotFound desc = could not find container \"7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373\": container with ID starting with 7062a8023dc2ead7420cb4dc85def511207adfbf7e6ae2475d55ae5605813373 not found: ID does not exist"
Nov 03 16:47:15.513202 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.513166    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"]
Nov 03 16:47:15.519241 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.519211    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-23c440-6576874c97-n966f"]
Nov 03 16:47:15.636363 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.636325    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"]
Nov 03 16:47:15.645966 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:15.645917    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-23c440-664949f775-kljqt"]
Nov 03 16:47:16.310479 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:16.310448    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="978db8fe-4aa9-40a6-8183-def3b49e2b68" path="/var/lib/kubelet/pods/978db8fe-4aa9-40a6-8183-def3b49e2b68/volumes"
Nov 03 16:47:16.310911 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:16.310896    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eade339c-4ff4-49d3-b634-1ad4459904f3" path="/var/lib/kubelet/pods/eade339c-4ff4-49d3-b634-1ad4459904f3/volumes"
Nov 03 16:47:16.636926 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:16.636894    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"]
Nov 03 16:47:16.754499 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:16.754463    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"]
Nov 03 16:47:18.095228 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.095192    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"]
Nov 03 16:47:18.095636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.095402    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb" podUID="16e5e2dc-9bb6-4120-9904-06e1e2da1f8b" containerName="maas-api" containerID="cri-o://bb20d2b5ddb5bb171d1b6c12f52062c735a805e465bbce6b55f4b3aa458d60b2" gracePeriod=30
Nov 03 16:47:18.380591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.379842    5605 generic.go:356] "Generic (PLEG): container finished" podID="16e5e2dc-9bb6-4120-9904-06e1e2da1f8b" containerID="bb20d2b5ddb5bb171d1b6c12f52062c735a805e465bbce6b55f4b3aa458d60b2" exitCode=0
Nov 03 16:47:18.380591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.379991    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb" event={"ID":"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b","Type":"ContainerDied","Data":"bb20d2b5ddb5bb171d1b6c12f52062c735a805e465bbce6b55f4b3aa458d60b2"}
Nov 03 16:47:18.380591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.380043    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb" event={"ID":"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b","Type":"ContainerDied","Data":"2f030d3b445971faad2a463ea7d35b180713b0c6041c02f922d3a12a7a2b3b87"}
Nov 03 16:47:18.380591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.380073    5605 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f030d3b445971faad2a463ea7d35b180713b0c6041c02f922d3a12a7a2b3b87"
Nov 03 16:47:18.382585 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.382260    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:47:18.546270 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.546235    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-maas-api-tls\") pod \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") "
Nov 03 16:47:18.546605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.546585    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-openshift-service-ca\") pod \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") "
Nov 03 16:47:18.546905 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.546884    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-kube-api-access-h4jh2\" (UniqueName: \"kubernetes.io/projected/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-kube-api-access-h4jh2\") pod \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") "
Nov 03 16:47:18.547128 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.547111    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-metrics-certs\") pod \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\" (UID: \"16e5e2dc-9bb6-4120-9904-06e1e2da1f8b\") "
Nov 03 16:47:18.548376 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.548348    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-openshift-service-ca" pod "16e5e2dc-9bb6-4120-9904-06e1e2da1f8b" (UID: "16e5e2dc-9bb6-4120-9904-06e1e2da1f8b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:47:18.559019 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.558976    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-kube-api-access-h4jh2" pod "16e5e2dc-9bb6-4120-9904-06e1e2da1f8b" (UID: "16e5e2dc-9bb6-4120-9904-06e1e2da1f8b"). InnerVolumeSpecName "kube-api-access-h4jh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:47:18.559692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.559660    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-metrics-certs" pod "16e5e2dc-9bb6-4120-9904-06e1e2da1f8b" (UID: "16e5e2dc-9bb6-4120-9904-06e1e2da1f8b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:47:18.560949 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.560920    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-maas-api-tls" pod "16e5e2dc-9bb6-4120-9904-06e1e2da1f8b" (UID: "16e5e2dc-9bb6-4120-9904-06e1e2da1f8b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:47:18.649192 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.649003    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:18.649421 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.649400    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:18.649529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.649517    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:18.649621 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:18.649610    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4jh2\" (UniqueName: \"kubernetes.io/projected/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b-kube-api-access-h4jh2\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:19.386675 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:19.386646    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"
Nov 03 16:47:19.562675 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:19.562636    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"]
Nov 03 16:47:19.612715 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:19.612678    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6f58cf-7d8fcd7c85-jnfwb"]
Nov 03 16:47:20.309959 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:20.309924    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16e5e2dc-9bb6-4120-9904-06e1e2da1f8b" path="/var/lib/kubelet/pods/16e5e2dc-9bb6-4120-9904-06e1e2da1f8b/volumes"
Nov 03 16:47:21.637741 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:21.637672    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x" podUID="89e47de6-1d71-460d-8323-34aa5222eb6b" containerName="payload-processing" containerID="cri-o://40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b" gracePeriod=25
Nov 03 16:47:21.755721 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:21.755680    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf" podUID="36f78ba2-86af-4225-ba2e-35f3d772e34a" containerName="payload-pre-processing" containerID="cri-o://40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee" gracePeriod=25
Nov 03 16:47:21.902455 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:21.902397    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:47:22.013454 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.013419    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89e47de6-1d71-460d-8323-34aa5222eb6b-kube-api-access-xtht9\" (UniqueName: \"kubernetes.io/projected/89e47de6-1d71-460d-8323-34aa5222eb6b-kube-api-access-xtht9\") pod \"89e47de6-1d71-460d-8323-34aa5222eb6b\" (UID: \"89e47de6-1d71-460d-8323-34aa5222eb6b\") "
Nov 03 16:47:22.013644 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.013571    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89e47de6-1d71-460d-8323-34aa5222eb6b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89e47de6-1d71-460d-8323-34aa5222eb6b-plugins-config-volume\") pod \"89e47de6-1d71-460d-8323-34aa5222eb6b\" (UID: \"89e47de6-1d71-460d-8323-34aa5222eb6b\") "
Nov 03 16:47:22.015053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.015018    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89e47de6-1d71-460d-8323-34aa5222eb6b-plugins-config-volume" pod "89e47de6-1d71-460d-8323-34aa5222eb6b" (UID: "89e47de6-1d71-460d-8323-34aa5222eb6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:47:22.016366 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.016342    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89e47de6-1d71-460d-8323-34aa5222eb6b-kube-api-access-xtht9" pod "89e47de6-1d71-460d-8323-34aa5222eb6b" (UID: "89e47de6-1d71-460d-8323-34aa5222eb6b"). InnerVolumeSpecName "kube-api-access-xtht9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:47:22.035072 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.035038    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:47:22.115713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.115671    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtht9\" (UniqueName: \"kubernetes.io/projected/89e47de6-1d71-460d-8323-34aa5222eb6b-kube-api-access-xtht9\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:22.115713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.115704    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89e47de6-1d71-460d-8323-34aa5222eb6b-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:22.216778 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.216679    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36f78ba2-86af-4225-ba2e-35f3d772e34a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36f78ba2-86af-4225-ba2e-35f3d772e34a-plugins-config-volume\") pod \"36f78ba2-86af-4225-ba2e-35f3d772e34a\" (UID: \"36f78ba2-86af-4225-ba2e-35f3d772e34a\") "
Nov 03 16:47:22.216974 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.216893    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36f78ba2-86af-4225-ba2e-35f3d772e34a-kube-api-access-4snbg\" (UniqueName: \"kubernetes.io/projected/36f78ba2-86af-4225-ba2e-35f3d772e34a-kube-api-access-4snbg\") pod \"36f78ba2-86af-4225-ba2e-35f3d772e34a\" (UID: \"36f78ba2-86af-4225-ba2e-35f3d772e34a\") "
Nov 03 16:47:22.217140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.217119    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36f78ba2-86af-4225-ba2e-35f3d772e34a-plugins-config-volume" pod "36f78ba2-86af-4225-ba2e-35f3d772e34a" (UID: "36f78ba2-86af-4225-ba2e-35f3d772e34a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:47:22.217422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.217405    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36f78ba2-86af-4225-ba2e-35f3d772e34a-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:22.219226 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.219196    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f78ba2-86af-4225-ba2e-35f3d772e34a-kube-api-access-4snbg" pod "36f78ba2-86af-4225-ba2e-35f3d772e34a" (UID: "36f78ba2-86af-4225-ba2e-35f3d772e34a"). InnerVolumeSpecName "kube-api-access-4snbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:47:22.318944 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.318915    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4snbg\" (UniqueName: \"kubernetes.io/projected/36f78ba2-86af-4225-ba2e-35f3d772e34a-kube-api-access-4snbg\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:22.406427 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.406390    5605 generic.go:356] "Generic (PLEG): container finished" podID="89e47de6-1d71-460d-8323-34aa5222eb6b" containerID="40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b" exitCode=0
Nov 03 16:47:22.406598 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.406439    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"
Nov 03 16:47:22.406598 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.406480    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x" event={"ID":"89e47de6-1d71-460d-8323-34aa5222eb6b","Type":"ContainerDied","Data":"40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b"}
Nov 03 16:47:22.406598 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.406536    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x" event={"ID":"89e47de6-1d71-460d-8323-34aa5222eb6b","Type":"ContainerDied","Data":"cd57343528ad5d46f76f00eb750981bfce245e9ab7746059946dea4dbb41c3d0"}
Nov 03 16:47:22.406598 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.406560    5605 scope.go:122] "RemoveContainer" containerID="40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b"
Nov 03 16:47:22.407959 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.407931    5605 generic.go:356] "Generic (PLEG): container finished" podID="36f78ba2-86af-4225-ba2e-35f3d772e34a" containerID="40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee" exitCode=0
Nov 03 16:47:22.408073 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.407980    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"
Nov 03 16:47:22.408073 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.407983    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf" event={"ID":"36f78ba2-86af-4225-ba2e-35f3d772e34a","Type":"ContainerDied","Data":"40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee"}
Nov 03 16:47:22.408073 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.408023    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf" event={"ID":"36f78ba2-86af-4225-ba2e-35f3d772e34a","Type":"ContainerDied","Data":"ce836f1838b6d8bb05dd8cbf2204ce016bbb4469a94d48df3cd07af00e627115"}
Nov 03 16:47:22.416790 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.416772    5605 scope.go:122] "RemoveContainer" containerID="40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b"
Nov 03 16:47:22.417072 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:22.417054    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b\": container with ID starting with 40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b not found: ID does not exist" containerID="40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b"
Nov 03 16:47:22.417136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.417078    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b"} err="failed to get container status \"40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b\": rpc error: code = NotFound desc = could not find container \"40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b\": container with ID starting with 40faba47e5097f732786c627f9f9a5a3d8b0867051ce1bd0390cf31526ec892b not found: ID does not exist"
Nov 03 16:47:22.417136 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.417093    5605 scope.go:122] "RemoveContainer" containerID="40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee"
Nov 03 16:47:22.429101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.429086    5605 scope.go:122] "RemoveContainer" containerID="40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee"
Nov 03 16:47:22.429359 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:22.429341    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee\": container with ID starting with 40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee not found: ID does not exist" containerID="40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee"
Nov 03 16:47:22.429409 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.429368    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee"} err="failed to get container status \"40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee\": rpc error: code = NotFound desc = could not find container \"40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee\": container with ID starting with 40d16be722e9320e5046ad5cc4634215e1896ad1d89cac133968b05f7dd1a7ee not found: ID does not exist"
Nov 03 16:47:22.581360 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.581266    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"]
Nov 03 16:47:22.586137 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.586107    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6f58cf-6796d7dbb7-vg92x"]
Nov 03 16:47:22.772765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.771796    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"]
Nov 03 16:47:22.775041 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:22.775018    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6f58cf-f8bd4b955-rpnkf"]
Nov 03 16:47:24.310051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:24.310021    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36f78ba2-86af-4225-ba2e-35f3d772e34a" path="/var/lib/kubelet/pods/36f78ba2-86af-4225-ba2e-35f3d772e34a/volumes"
Nov 03 16:47:24.310422 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:24.310408    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89e47de6-1d71-460d-8323-34aa5222eb6b" path="/var/lib/kubelet/pods/89e47de6-1d71-460d-8323-34aa5222eb6b/volumes"
Nov 03 16:47:33.393418 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:33.393374    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:47:37.992284 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:37.992246    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"]
Nov 03 16:47:38.043325 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:38.043294    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"]
Nov 03 16:47:40.510992 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.510949    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"]
Nov 03 16:47:40.511521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511504    5605 cpu_manager.go:414] "containerMap: removing container" podUID="c2e4bfb6-f892-4dae-8e86-36af9c6def94" containerName="payload-processing"
Nov 03 16:47:40.511521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511522    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2e4bfb6-f892-4dae-8e86-36af9c6def94" containerName="payload-processing"
Nov 03 16:47:40.511605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511539    5605 cpu_manager.go:414] "containerMap: removing container" podUID="8ea72c89-5ec4-4822-be54-be0b1288d62c" containerName="payload-pre-processing"
Nov 03 16:47:40.511605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511545    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ea72c89-5ec4-4822-be54-be0b1288d62c" containerName="payload-pre-processing"
Nov 03 16:47:40.511605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511559    5605 cpu_manager.go:414] "containerMap: removing container" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy"
Nov 03 16:47:40.511605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511564    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy"
Nov 03 16:47:40.511605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511575    5605 cpu_manager.go:414] "containerMap: removing container" podUID="978db8fe-4aa9-40a6-8183-def3b49e2b68" containerName="payload-pre-processing"
Nov 03 16:47:40.511605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511582    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="978db8fe-4aa9-40a6-8183-def3b49e2b68" containerName="payload-pre-processing"
Nov 03 16:47:40.511605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511593    5605 cpu_manager.go:414] "containerMap: removing container" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy"
Nov 03 16:47:40.511605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511598    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511613    5605 cpu_manager.go:414] "containerMap: removing container" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511618    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511626    5605 cpu_manager.go:414] "containerMap: removing container" podUID="16e5e2dc-9bb6-4120-9904-06e1e2da1f8b" containerName="maas-api"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511630    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16e5e2dc-9bb6-4120-9904-06e1e2da1f8b" containerName="maas-api"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511638    5605 cpu_manager.go:414] "containerMap: removing container" podUID="89e47de6-1d71-460d-8323-34aa5222eb6b" containerName="payload-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511642    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89e47de6-1d71-460d-8323-34aa5222eb6b" containerName="payload-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511652    5605 cpu_manager.go:414] "containerMap: removing container" podUID="36f78ba2-86af-4225-ba2e-35f3d772e34a" containerName="payload-pre-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511656    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36f78ba2-86af-4225-ba2e-35f3d772e34a" containerName="payload-pre-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511666    5605 cpu_manager.go:414] "containerMap: removing container" podUID="eade339c-4ff4-49d3-b634-1ad4459904f3" containerName="payload-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511670    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eade339c-4ff4-49d3-b634-1ad4459904f3" containerName="payload-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511680    5605 cpu_manager.go:414] "containerMap: removing container" podUID="9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" containerName="main"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511684    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" containerName="main"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511778    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6e016ec-3eaf-4d13-a322-425d83281dc0" containerName="istio-proxy"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511786    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="16e5e2dc-9bb6-4120-9904-06e1e2da1f8b" containerName="maas-api"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511794    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="89e47de6-1d71-460d-8323-34aa5222eb6b" containerName="payload-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511802    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="36f78ba2-86af-4225-ba2e-35f3d772e34a" containerName="payload-pre-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511809    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="eade339c-4ff4-49d3-b634-1ad4459904f3" containerName="payload-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511814    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="9cd488c1-cb1d-42e0-bec6-be52b44f1c1f" containerName="main"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511821    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2e4bfb6-f892-4dae-8e86-36af9c6def94" containerName="payload-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511828    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ea72c89-5ec4-4822-be54-be0b1288d62c" containerName="payload-pre-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511833    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="266192f8-6a4e-4736-b400-22433bd0e632" containerName="istio-proxy"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511839    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="978db8fe-4aa9-40a6-8183-def3b49e2b68" containerName="payload-pre-processing"
Nov 03 16:47:40.511910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.511845    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="03b5a7fa-c4c4-4778-a652-09969f86569e" containerName="istio-proxy"
Nov 03 16:47:40.516662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.516641    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.576396 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.576358    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"]
Nov 03 16:47:40.592043 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.592003    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-maas-api-tls\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.592213 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.592162    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpg47\" (UniqueName: \"kubernetes.io/projected/62c29f8b-bd93-4ab0-ac1d-682247836f51-kube-api-access-zpg47\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.592302 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.592253    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-metrics-certs\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.592418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.592399    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c29f8b-bd93-4ab0-ac1d-682247836f51-openshift-service-ca\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.622713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.622681    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c626cd-serving-cert\""
Nov 03 16:47:40.694140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.694105    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-maas-api-tls\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.694345 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.694183    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpg47\" (UniqueName: \"kubernetes.io/projected/62c29f8b-bd93-4ab0-ac1d-682247836f51-kube-api-access-zpg47\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.694345 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.694224    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-metrics-certs\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.694345 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:40.694253    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-serving-cert: secret "maas-api-e2e-api-a-c626cd-serving-cert" not found
Nov 03 16:47:40.694345 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:40.694310    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-maas-api-tls podName:62c29f8b-bd93-4ab0-ac1d-682247836f51 nodeName:}" failed. No retries permitted until 2026-11-03 16:47:41.194295268 +0000 UTC m=+2147.018746541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-maas-api-tls") pod "maas-api-e2e-api-a-c626cd-58cf775875-qwq8q" (UID: "62c29f8b-bd93-4ab0-ac1d-682247836f51") : secret "maas-api-e2e-api-a-c626cd-serving-cert" not found
Nov 03 16:47:40.694602 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.694362    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c29f8b-bd93-4ab0-ac1d-682247836f51-openshift-service-ca\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.695004 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.694982    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c29f8b-bd93-4ab0-ac1d-682247836f51-openshift-service-ca\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.696756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.696739    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-metrics-certs\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:40.784859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:40.784693    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpg47\" (UniqueName: \"kubernetes.io/projected/62c29f8b-bd93-4ab0-ac1d-682247836f51-kube-api-access-zpg47\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:41.202779 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:41.202745    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-maas-api-tls\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:41.205220 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:41.205184    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-maas-api-tls\") pod \"maas-api-e2e-api-a-c626cd-58cf775875-qwq8q\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:41.430758 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:41.430706    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:41.818897 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:41.818862    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"]
Nov 03 16:47:41.820221 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:47:41.820193    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c29f8b_bd93_4ab0_ac1d_682247836f51.slice/crio-0d830bc4c07f383b5493bd14f7b26e151441cef4d7c00bb9b0ebc73853934c37 WatchSource:0}: Error finding container 0d830bc4c07f383b5493bd14f7b26e151441cef4d7c00bb9b0ebc73853934c37: Status 404 returned error can't find the container with id 0d830bc4c07f383b5493bd14f7b26e151441cef4d7c00bb9b0ebc73853934c37
Nov 03 16:47:41.822013 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:41.821996    5605 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 16:47:42.532389 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:42.532353    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q" event={"ID":"62c29f8b-bd93-4ab0-ac1d-682247836f51","Type":"ContainerStarted","Data":"0d830bc4c07f383b5493bd14f7b26e151441cef4d7c00bb9b0ebc73853934c37"}
Nov 03 16:47:43.540705 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:43.540669    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q" event={"ID":"62c29f8b-bd93-4ab0-ac1d-682247836f51","Type":"ContainerStarted","Data":"9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe"}
Nov 03 16:47:43.541134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:43.540757    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:43.662961 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:43.662910    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q" podStartSLOduration=2.686313953 podStartE2EDuration="3.662897787s" totalImagesPullingTime="976.583834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:47:43.662431864 +0000 UTC m=+2149.486883134" watchObservedRunningTime="2026-11-03 16:47:43.662897787 +0000 UTC m=+2149.487349057"
Nov 03 16:47:49.553413 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:49.553383    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:47:53.013663 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.013602    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main" containerID="cri-o://030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8" gracePeriod=45
Nov 03 16:47:53.063878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.063818    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" containerID="cri-o://b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54" gracePeriod=45
Nov 03 16:47:53.249085 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.249044    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": dial tcp 10.133.0.39:8000: connect: connection refused"
Nov 03 16:47:53.344914 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.344882    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:47:53.376869 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.376845    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:47:53.469571 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.469539    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tmp-dir\") pod \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") "
Nov 03 16:47:53.469787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.469634    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-home\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-home\") pod \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") "
Nov 03 16:47:53.469787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.469709    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-dshm\") pod \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") "
Nov 03 16:47:53.469964 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.469910    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tmp-dir" pod "464d2b0b-6212-43b3-b182-7f860bfe2e4f" (UID: "464d2b0b-6212-43b3-b182-7f860bfe2e4f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:53.470028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.469979    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-home" pod "464d2b0b-6212-43b3-b182-7f860bfe2e4f" (UID: "464d2b0b-6212-43b3-b182-7f860bfe2e4f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:53.470188 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.470170    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-dshm\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-dshm\") pod \"075e5aa1-a616-42dc-8c95-62f8db2c1194\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") "
Nov 03 16:47:53.470261 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.470231    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/075e5aa1-a616-42dc-8c95-62f8db2c1194-kube-api-access-fg999\" (UniqueName: \"kubernetes.io/projected/075e5aa1-a616-42dc-8c95-62f8db2c1194-kube-api-access-fg999\") pod \"075e5aa1-a616-42dc-8c95-62f8db2c1194\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") "
Nov 03 16:47:53.470325 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.470286    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-tmp-dir\") pod \"075e5aa1-a616-42dc-8c95-62f8db2c1194\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") "
Nov 03 16:47:53.470390 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.470343    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/075e5aa1-a616-42dc-8c95-62f8db2c1194-tls-certs\" (UniqueName: \"kubernetes.io/secret/075e5aa1-a616-42dc-8c95-62f8db2c1194-tls-certs\") pod \"075e5aa1-a616-42dc-8c95-62f8db2c1194\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") "
Nov 03 16:47:53.470607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.470409    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tls-certs\" (UniqueName: \"kubernetes.io/secret/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tls-certs\") pod \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") "
Nov 03 16:47:53.470607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.470444    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-model-cache\") pod \"075e5aa1-a616-42dc-8c95-62f8db2c1194\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") "
Nov 03 16:47:53.470607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.470495    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-home\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-home\") pod \"075e5aa1-a616-42dc-8c95-62f8db2c1194\" (UID: \"075e5aa1-a616-42dc-8c95-62f8db2c1194\") "
Nov 03 16:47:53.470607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.470530    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/464d2b0b-6212-43b3-b182-7f860bfe2e4f-kube-api-access-q4xg9\" (UniqueName: \"kubernetes.io/projected/464d2b0b-6212-43b3-b182-7f860bfe2e4f-kube-api-access-q4xg9\") pod \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") "
Nov 03 16:47:53.470864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.470632    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-model-cache\") pod \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\" (UID: \"464d2b0b-6212-43b3-b182-7f860bfe2e4f\") "
Nov 03 16:47:53.471084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.471013    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-home" pod "075e5aa1-a616-42dc-8c95-62f8db2c1194" (UID: "075e5aa1-a616-42dc-8c95-62f8db2c1194"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:53.471259 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.471219    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-tmp-dir" pod "075e5aa1-a616-42dc-8c95-62f8db2c1194" (UID: "075e5aa1-a616-42dc-8c95-62f8db2c1194"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:53.471585 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.471483    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-model-cache" pod "464d2b0b-6212-43b3-b182-7f860bfe2e4f" (UID: "464d2b0b-6212-43b3-b182-7f860bfe2e4f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:53.471585 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.471552    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-model-cache" pod "075e5aa1-a616-42dc-8c95-62f8db2c1194" (UID: "075e5aa1-a616-42dc-8c95-62f8db2c1194"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:53.472446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.472424    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.472569 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.472556    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.472665 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.472653    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.472807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.472792    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.472906 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.472893    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.473002 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.472989    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.473895 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.473862    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tls-certs" pod "464d2b0b-6212-43b3-b182-7f860bfe2e4f" (UID: "464d2b0b-6212-43b3-b182-7f860bfe2e4f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:47:53.473982 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.473901    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/075e5aa1-a616-42dc-8c95-62f8db2c1194-kube-api-access-fg999" pod "075e5aa1-a616-42dc-8c95-62f8db2c1194" (UID: "075e5aa1-a616-42dc-8c95-62f8db2c1194"). InnerVolumeSpecName "kube-api-access-fg999". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:47:53.474037 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.474012    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-dshm" pod "464d2b0b-6212-43b3-b182-7f860bfe2e4f" (UID: "464d2b0b-6212-43b3-b182-7f860bfe2e4f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:53.475048 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.475019    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-dshm" pod "075e5aa1-a616-42dc-8c95-62f8db2c1194" (UID: "075e5aa1-a616-42dc-8c95-62f8db2c1194"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:47:53.475350 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.475319    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/075e5aa1-a616-42dc-8c95-62f8db2c1194-tls-certs" pod "075e5aa1-a616-42dc-8c95-62f8db2c1194" (UID: "075e5aa1-a616-42dc-8c95-62f8db2c1194"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:47:53.475832 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.475803    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/464d2b0b-6212-43b3-b182-7f860bfe2e4f-kube-api-access-q4xg9" pod "464d2b0b-6212-43b3-b182-7f860bfe2e4f" (UID: "464d2b0b-6212-43b3-b182-7f860bfe2e4f"). InnerVolumeSpecName "kube-api-access-q4xg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:47:53.577842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.574004    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4xg9\" (UniqueName: \"kubernetes.io/projected/464d2b0b-6212-43b3-b182-7f860bfe2e4f-kube-api-access-q4xg9\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.577842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.574040    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/464d2b0b-6212-43b3-b182-7f860bfe2e4f-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.577842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.574055    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/075e5aa1-a616-42dc-8c95-62f8db2c1194-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.577842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.574067    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg999\" (UniqueName: \"kubernetes.io/projected/075e5aa1-a616-42dc-8c95-62f8db2c1194-kube-api-access-fg999\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.577842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.574082    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/075e5aa1-a616-42dc-8c95-62f8db2c1194-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.577842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.574095    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/464d2b0b-6212-43b3-b182-7f860bfe2e4f-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:47:53.636986 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.636946    5605 generic.go:356] "Generic (PLEG): container finished" podID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerID="030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8" exitCode=0
Nov 03 16:47:53.637166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.636990    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"
Nov 03 16:47:53.637166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.637027    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" event={"ID":"075e5aa1-a616-42dc-8c95-62f8db2c1194","Type":"ContainerDied","Data":"030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8"}
Nov 03 16:47:53.637166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.637080    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6" event={"ID":"075e5aa1-a616-42dc-8c95-62f8db2c1194","Type":"ContainerDied","Data":"f61f88dec809422971defe4d91b601b622da8992f24aa0ee516776eb508cddcd"}
Nov 03 16:47:53.637166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.637100    5605 scope.go:122] "RemoveContainer" containerID="030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8"
Nov 03 16:47:53.638642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.638610    5605 generic.go:356] "Generic (PLEG): container finished" podID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerID="b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54" exitCode=0
Nov 03 16:47:53.638780 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.638654    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"
Nov 03 16:47:53.638780 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.638686    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" event={"ID":"464d2b0b-6212-43b3-b182-7f860bfe2e4f","Type":"ContainerDied","Data":"b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54"}
Nov 03 16:47:53.638780 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.638751    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw" event={"ID":"464d2b0b-6212-43b3-b182-7f860bfe2e4f","Type":"ContainerDied","Data":"5fb50fe8b5e971272143835bf0d3e117bd6f5d49a45d74e0fdf9f22f7e76f07b"}
Nov 03 16:47:53.648143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.648123    5605 scope.go:122] "RemoveContainer" containerID="030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8"
Nov 03 16:47:53.648468 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:53.648447    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8\": container with ID starting with 030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8 not found: ID does not exist" containerID="030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8"
Nov 03 16:47:53.648562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.648480    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8"} err="failed to get container status \"030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8\": rpc error: code = NotFound desc = could not find container \"030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8\": container with ID starting with 030b8617cf065c663053ba6270a4e7daf049eb27f713e630925ce849c37cc1b8 not found: ID does not exist"
Nov 03 16:47:53.648562 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.648503    5605 scope.go:122] "RemoveContainer" containerID="b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54"
Nov 03 16:47:53.657937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.657919    5605 scope.go:122] "RemoveContainer" containerID="b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54"
Nov 03 16:47:53.658200 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:47:53.658181    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54\": container with ID starting with b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54 not found: ID does not exist" containerID="b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54"
Nov 03 16:47:53.658261 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.658210    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54"} err="failed to get container status \"b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54\": rpc error: code = NotFound desc = could not find container \"b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54\": container with ID starting with b3f5f228d0718330b3b6430af2e91103897455780989e5ece99c04e02a0c8d54 not found: ID does not exist"
Nov 03 16:47:53.811280 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.811243    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"]
Nov 03 16:47:53.813789 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.813770    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/premium-w4-2bb620-kserve-74dd78bc88-s6tl6"]
Nov 03 16:47:53.929504 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.929469    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"]
Nov 03 16:47:53.934881 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:53.934857    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-2bb620/simulator-w4-2bb620-kserve-75c9994b57-zclvw"]
Nov 03 16:47:54.319521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:54.319444    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" path="/var/lib/kubelet/pods/075e5aa1-a616-42dc-8c95-62f8db2c1194/volumes"
Nov 03 16:47:54.320137 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:47:54.320117    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" path="/var/lib/kubelet/pods/464d2b0b-6212-43b3-b182-7f860bfe2e4f/volumes"
Nov 03 16:48:03.394024 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:48:03.393983    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:48:15.482470 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.482417    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"]
Nov 03 16:48:15.482982 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.482667    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn" podUID="c62b1d42-affc-4890-b8fa-7d044b2bba96" containerName="maas-api" containerID="cri-o://c499d9a432b020b64e2e853a2276c6ce0fc4f70f302b079429b6e61cbcbb18b5" gracePeriod=30
Nov 03 16:48:15.790482 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.790454    5605 generic.go:356] "Generic (PLEG): container finished" podID="c62b1d42-affc-4890-b8fa-7d044b2bba96" containerID="c499d9a432b020b64e2e853a2276c6ce0fc4f70f302b079429b6e61cbcbb18b5" exitCode=0
Nov 03 16:48:15.790605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.790524    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn" event={"ID":"c62b1d42-affc-4890-b8fa-7d044b2bba96","Type":"ContainerDied","Data":"c499d9a432b020b64e2e853a2276c6ce0fc4f70f302b079429b6e61cbcbb18b5"}
Nov 03 16:48:15.812605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.812580    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:48:15.973908 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.973878    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-maas-api-tls\") pod \"c62b1d42-affc-4890-b8fa-7d044b2bba96\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") "
Nov 03 16:48:15.974094 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.973928    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c62b1d42-affc-4890-b8fa-7d044b2bba96-kube-api-access-g67d8\" (UniqueName: \"kubernetes.io/projected/c62b1d42-affc-4890-b8fa-7d044b2bba96-kube-api-access-g67d8\") pod \"c62b1d42-affc-4890-b8fa-7d044b2bba96\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") "
Nov 03 16:48:15.974094 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.974054    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c62b1d42-affc-4890-b8fa-7d044b2bba96-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c62b1d42-affc-4890-b8fa-7d044b2bba96-openshift-service-ca\") pod \"c62b1d42-affc-4890-b8fa-7d044b2bba96\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") "
Nov 03 16:48:15.974230 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.974100    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-metrics-certs\") pod \"c62b1d42-affc-4890-b8fa-7d044b2bba96\" (UID: \"c62b1d42-affc-4890-b8fa-7d044b2bba96\") "
Nov 03 16:48:15.974507 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.974468    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c62b1d42-affc-4890-b8fa-7d044b2bba96-openshift-service-ca" pod "c62b1d42-affc-4890-b8fa-7d044b2bba96" (UID: "c62b1d42-affc-4890-b8fa-7d044b2bba96"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:48:15.976347 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.976318    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-metrics-certs" pod "c62b1d42-affc-4890-b8fa-7d044b2bba96" (UID: "c62b1d42-affc-4890-b8fa-7d044b2bba96"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:48:15.976463 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.976409    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c62b1d42-affc-4890-b8fa-7d044b2bba96-kube-api-access-g67d8" pod "c62b1d42-affc-4890-b8fa-7d044b2bba96" (UID: "c62b1d42-affc-4890-b8fa-7d044b2bba96"). InnerVolumeSpecName "kube-api-access-g67d8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:48:15.976672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:15.976658    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-maas-api-tls" pod "c62b1d42-affc-4890-b8fa-7d044b2bba96" (UID: "c62b1d42-affc-4890-b8fa-7d044b2bba96"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:48:16.076263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:16.076116    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:16.076263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:16.076156    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g67d8\" (UniqueName: \"kubernetes.io/projected/c62b1d42-affc-4890-b8fa-7d044b2bba96-kube-api-access-g67d8\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:16.076263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:16.076172    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c62b1d42-affc-4890-b8fa-7d044b2bba96-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:16.076263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:16.076185    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c62b1d42-affc-4890-b8fa-7d044b2bba96-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:16.798187 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:16.798156    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"
Nov 03 16:48:16.798187 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:16.798178    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn" event={"ID":"c62b1d42-affc-4890-b8fa-7d044b2bba96","Type":"ContainerDied","Data":"1bbd22910710a80482b87446303581ee037803d6af0992690236016a75492896"}
Nov 03 16:48:16.798827 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:16.798272    5605 scope.go:122] "RemoveContainer" containerID="c499d9a432b020b64e2e853a2276c6ce0fc4f70f302b079429b6e61cbcbb18b5"
Nov 03 16:48:16.971471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:16.971439    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"]
Nov 03 16:48:17.022164 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:17.022130    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23c440-76fd77d444-454dn"]
Nov 03 16:48:18.309958 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:18.309920    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c62b1d42-affc-4890-b8fa-7d044b2bba96" path="/var/lib/kubelet/pods/c62b1d42-affc-4890-b8fa-7d044b2bba96/volumes"
Nov 03 16:48:21.744602 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:21.744561    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"]
Nov 03 16:48:21.745158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:21.744834    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs" podUID="8b3dae34-f264-44fd-8aae-725e3905595a" containerName="maas-api" containerID="cri-o://978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d" gracePeriod=30
Nov 03 16:48:22.066118 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.066092    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:48:22.173281 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.173241    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b3dae34-f264-44fd-8aae-725e3905595a-kube-api-access-tbc25\" (UniqueName: \"kubernetes.io/projected/8b3dae34-f264-44fd-8aae-725e3905595a-kube-api-access-tbc25\") pod \"8b3dae34-f264-44fd-8aae-725e3905595a\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") "
Nov 03 16:48:22.173490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.173324    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b3dae34-f264-44fd-8aae-725e3905595a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b3dae34-f264-44fd-8aae-725e3905595a-openshift-service-ca\") pod \"8b3dae34-f264-44fd-8aae-725e3905595a\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") "
Nov 03 16:48:22.173490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.173397    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-metrics-certs\") pod \"8b3dae34-f264-44fd-8aae-725e3905595a\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") "
Nov 03 16:48:22.173610 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.173504    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-maas-api-tls\") pod \"8b3dae34-f264-44fd-8aae-725e3905595a\" (UID: \"8b3dae34-f264-44fd-8aae-725e3905595a\") "
Nov 03 16:48:22.192957 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.192924    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b3dae34-f264-44fd-8aae-725e3905595a-openshift-service-ca" pod "8b3dae34-f264-44fd-8aae-725e3905595a" (UID: "8b3dae34-f264-44fd-8aae-725e3905595a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:48:22.204680 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.204618    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-metrics-certs" pod "8b3dae34-f264-44fd-8aae-725e3905595a" (UID: "8b3dae34-f264-44fd-8aae-725e3905595a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:48:22.205620 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.205590    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-maas-api-tls" pod "8b3dae34-f264-44fd-8aae-725e3905595a" (UID: "8b3dae34-f264-44fd-8aae-725e3905595a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:48:22.214671 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.214620    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b3dae34-f264-44fd-8aae-725e3905595a-kube-api-access-tbc25" pod "8b3dae34-f264-44fd-8aae-725e3905595a" (UID: "8b3dae34-f264-44fd-8aae-725e3905595a"). InnerVolumeSpecName "kube-api-access-tbc25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:48:22.274996 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.274968    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbc25\" (UniqueName: \"kubernetes.io/projected/8b3dae34-f264-44fd-8aae-725e3905595a-kube-api-access-tbc25\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:22.274996 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.274995    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b3dae34-f264-44fd-8aae-725e3905595a-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:22.275206 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.275006    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:22.275206 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.275015    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b3dae34-f264-44fd-8aae-725e3905595a-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:22.837955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.837917    5605 generic.go:356] "Generic (PLEG): container finished" podID="8b3dae34-f264-44fd-8aae-725e3905595a" containerID="978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d" exitCode=0
Nov 03 16:48:22.838354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.838075    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs" event={"ID":"8b3dae34-f264-44fd-8aae-725e3905595a","Type":"ContainerDied","Data":"978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d"}
Nov 03 16:48:22.838354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.838114    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs" event={"ID":"8b3dae34-f264-44fd-8aae-725e3905595a","Type":"ContainerDied","Data":"66246be655caf57859d64ea7808c0dcf0afd4604c43c258afa081f0ac75750d5"}
Nov 03 16:48:22.838354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.838134    5605 scope.go:122] "RemoveContainer" containerID="978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d"
Nov 03 16:48:22.838354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.838248    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"
Nov 03 16:48:22.849686 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.849655    5605 scope.go:122] "RemoveContainer" containerID="978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d"
Nov 03 16:48:22.850054 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:48:22.850031    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d\": container with ID starting with 978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d not found: ID does not exist" containerID="978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d"
Nov 03 16:48:22.850132 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:22.850065    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d"} err="failed to get container status \"978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d\": rpc error: code = NotFound desc = could not find container \"978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d\": container with ID starting with 978cf5466cb866860d77718160d15662c7a1a2113c7d0d4baf702ac053d0208d not found: ID does not exist"
Nov 03 16:48:23.010997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:23.010962    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"]
Nov 03 16:48:23.062131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:23.062057    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cc4dcd-7d5b6d4bfb-flfcs"]
Nov 03 16:48:24.310524 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:24.310492    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b3dae34-f264-44fd-8aae-725e3905595a" path="/var/lib/kubelet/pods/8b3dae34-f264-44fd-8aae-725e3905595a/volumes"
Nov 03 16:48:26.603011 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:26.602976    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"]
Nov 03 16:48:26.603521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:26.603260    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy" containerID="cri-o://6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6" gracePeriod=30
Nov 03 16:48:31.864487 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.864423    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:48:31.893460 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.893429    5605 generic.go:356] "Generic (PLEG): container finished" podID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerID="6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6" exitCode=0
Nov 03 16:48:31.893601 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.893484    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"
Nov 03 16:48:31.893601 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.893496    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" event={"ID":"5ba93ef4-459e-40b1-865d-fa5234b41a10","Type":"ContainerDied","Data":"6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6"}
Nov 03 16:48:31.893601 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.893563    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb" event={"ID":"5ba93ef4-459e-40b1-865d-fa5234b41a10","Type":"ContainerDied","Data":"c53b0e811446c2af3dad224adcf95397903e04efa84ba7abb32b110403ac5110"}
Nov 03 16:48:31.893601 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.893587    5605 scope.go:122] "RemoveContainer" containerID="6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6"
Nov 03 16:48:31.903265 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.903245    5605 scope.go:122] "RemoveContainer" containerID="079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13"
Nov 03 16:48:31.908549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908525    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-credential-socket\") pod \"5ba93ef4-459e-40b1-865d-fa5234b41a10\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") "
Nov 03 16:48:31.908656 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908566    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-certs\") pod \"5ba93ef4-459e-40b1-865d-fa5234b41a10\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") "
Nov 03 16:48:31.908656 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908601    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-envoy\") pod \"5ba93ef4-459e-40b1-865d-fa5234b41a10\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") "
Nov 03 16:48:31.908656 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908627    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-token\" (UniqueName: \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-token\") pod \"5ba93ef4-459e-40b1-865d-fa5234b41a10\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") "
Nov 03 16:48:31.908866 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908660    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-podinfo\") pod \"5ba93ef4-459e-40b1-865d-fa5234b41a10\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") "
Nov 03 16:48:31.908866 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908740    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ba93ef4-459e-40b1-865d-fa5234b41a10-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ba93ef4-459e-40b1-865d-fa5234b41a10-istiod-ca-cert\") pod \"5ba93ef4-459e-40b1-865d-fa5234b41a10\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") "
Nov 03 16:48:31.908866 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908854    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-data\") pod \"5ba93ef4-459e-40b1-865d-fa5234b41a10\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") "
Nov 03 16:48:31.908866 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908863    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-credential-socket" pod "5ba93ef4-459e-40b1-865d-fa5234b41a10" (UID: "5ba93ef4-459e-40b1-865d-fa5234b41a10"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:31.909091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908891    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-certs" pod "5ba93ef4-459e-40b1-865d-fa5234b41a10" (UID: "5ba93ef4-459e-40b1-865d-fa5234b41a10"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:31.909091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908902    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-kube-api-access-c9hxf\" (UniqueName: \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-kube-api-access-c9hxf\") pod \"5ba93ef4-459e-40b1-865d-fa5234b41a10\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") "
Nov 03 16:48:31.909091 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.908971    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-socket\") pod \"5ba93ef4-459e-40b1-865d-fa5234b41a10\" (UID: \"5ba93ef4-459e-40b1-865d-fa5234b41a10\") "
Nov 03 16:48:31.909375 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.909348    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-data" pod "5ba93ef4-459e-40b1-865d-fa5234b41a10" (UID: "5ba93ef4-459e-40b1-865d-fa5234b41a10"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:31.909557 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.909522    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-socket" pod "5ba93ef4-459e-40b1-865d-fa5234b41a10" (UID: "5ba93ef4-459e-40b1-865d-fa5234b41a10"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:31.909771 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.909695    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:31.909771 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.909718    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:31.909963 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.909773    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:31.909963 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.909788    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:31.910314 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.910285    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ba93ef4-459e-40b1-865d-fa5234b41a10-istiod-ca-cert" pod "5ba93ef4-459e-40b1-865d-fa5234b41a10" (UID: "5ba93ef4-459e-40b1-865d-fa5234b41a10"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:48:31.911128 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.911100    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-envoy" pod "5ba93ef4-459e-40b1-865d-fa5234b41a10" (UID: "5ba93ef4-459e-40b1-865d-fa5234b41a10"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:31.911489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.911467    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-kube-api-access-c9hxf" pod "5ba93ef4-459e-40b1-865d-fa5234b41a10" (UID: "5ba93ef4-459e-40b1-865d-fa5234b41a10"). InnerVolumeSpecName "kube-api-access-c9hxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:48:31.911556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.911542    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-token" pod "5ba93ef4-459e-40b1-865d-fa5234b41a10" (UID: "5ba93ef4-459e-40b1-865d-fa5234b41a10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:48:31.911615 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.911573    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-podinfo" pod "5ba93ef4-459e-40b1-865d-fa5234b41a10" (UID: "5ba93ef4-459e-40b1-865d-fa5234b41a10"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:48:31.912987 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.912969    5605 scope.go:122] "RemoveContainer" containerID="6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6"
Nov 03 16:48:31.913247 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:48:31.913228    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6\": container with ID starting with 6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6 not found: ID does not exist" containerID="6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6"
Nov 03 16:48:31.913308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.913255    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6"} err="failed to get container status \"6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6\": rpc error: code = NotFound desc = could not find container \"6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6\": container with ID starting with 6b351018c5a3b1b49c6b5cbe2951ea72704385eaf9258e88b6b140ced4b983d6 not found: ID does not exist"
Nov 03 16:48:31.913308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.913272    5605 scope.go:122] "RemoveContainer" containerID="079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13"
Nov 03 16:48:31.913498 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:48:31.913481    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13\": container with ID starting with 079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13 not found: ID does not exist" containerID="079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13"
Nov 03 16:48:31.913613 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:31.913503    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13"} err="failed to get container status \"079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13\": rpc error: code = NotFound desc = could not find container \"079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13\": container with ID starting with 079e6998db71af7ea01c8b0a2d45a42bd02a8dc7e3753daa9cb90667a6b58f13 not found: ID does not exist"
Nov 03 16:48:32.011582 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:32.011545    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9hxf\" (UniqueName: \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-kube-api-access-c9hxf\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:32.011582 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:32.011581    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:32.011807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:32.011591    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:32.011807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:32.011602    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ba93ef4-459e-40b1-865d-fa5234b41a10-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:32.011807 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:32.011611    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ba93ef4-459e-40b1-865d-fa5234b41a10-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:32.322793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:32.322761    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"]
Nov 03 16:48:32.329268 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:32.329240    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-23c440-openshift-default-58d96c55f-rgstb"]
Nov 03 16:48:33.394681 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:48:33.394648    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:48:34.308662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:34.308629    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" path="/var/lib/kubelet/pods/5ba93ef4-459e-40b1-865d-fa5234b41a10/volumes"
Nov 03 16:48:43.279330 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.279295    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"]
Nov 03 16:48:43.279804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.279511    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy" containerID="cri-o://f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb" gracePeriod=30
Nov 03 16:48:43.417489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.417412    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"]
Nov 03 16:48:43.426381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.418514    5605 cpu_manager.go:414] "containerMap: removing container" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy"
Nov 03 16:48:43.426625 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.426602    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy"
Nov 03 16:48:43.426778 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.426762    5605 cpu_manager.go:414] "containerMap: removing container" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main"
Nov 03 16:48:43.426905 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.426889    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main"
Nov 03 16:48:43.427239 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.427222    5605 cpu_manager.go:414] "containerMap: removing container" podUID="c62b1d42-affc-4890-b8fa-7d044b2bba96" containerName="maas-api"
Nov 03 16:48:43.427382 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.427369    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c62b1d42-affc-4890-b8fa-7d044b2bba96" containerName="maas-api"
Nov 03 16:48:43.427480 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.427470    5605 cpu_manager.go:414] "containerMap: removing container" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main"
Nov 03 16:48:43.427555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.427545    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main"
Nov 03 16:48:43.427631 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.427622    5605 cpu_manager.go:414] "containerMap: removing container" podUID="8b3dae34-f264-44fd-8aae-725e3905595a" containerName="maas-api"
Nov 03 16:48:43.427697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.427687    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b3dae34-f264-44fd-8aae-725e3905595a" containerName="maas-api"
Nov 03 16:48:43.430245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.430225    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="c62b1d42-affc-4890-b8fa-7d044b2bba96" containerName="maas-api"
Nov 03 16:48:43.430387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.430375    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="075e5aa1-a616-42dc-8c95-62f8db2c1194" containerName="main"
Nov 03 16:48:43.430479 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.430468    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b3dae34-f264-44fd-8aae-725e3905595a" containerName="maas-api"
Nov 03 16:48:43.430577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.430565    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy"
Nov 03 16:48:43.430689 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.430673    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy"
Nov 03 16:48:43.430805 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.430793    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="464d2b0b-6212-43b3-b182-7f860bfe2e4f" containerName="main"
Nov 03 16:48:43.442099 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.442038    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.481011 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.480978    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"]
Nov 03 16:48:43.485828 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.482092    5605 cpu_manager.go:414] "containerMap: removing container" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy"
Nov 03 16:48:43.485828 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.482795    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ba93ef4-459e-40b1-865d-fa5234b41a10" containerName="istio-proxy"
Nov 03 16:48:43.487910 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.487882    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"]
Nov 03 16:48:43.488063 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.488024    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.492563 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.492541    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"]
Nov 03 16:48:43.492707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.492647    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.530218 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.517571    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"]
Nov 03 16:48:43.548792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.543815    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"]
Nov 03 16:48:43.589912 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.589881    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.590070 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.589933    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.590070 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590019    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.590185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590066    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.590185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590142    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b444a3c-d720-476d-81b9-b4e15407c2e3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.590185 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590180    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.590354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590201    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.590354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590259    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.590354 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590305    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.590521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590379    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.590521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590439    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7twl7\" (UniqueName: \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-kube-api-access-7twl7\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.590521 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590468    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.590638 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590519    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.590638 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590552    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.590706 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590669    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.590782 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590710    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.590825 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590793    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.590856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590840    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.590889 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590868    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.590934 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590922    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.590968 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590955    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.591005 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.590979    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.591048 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.591029    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwxg7\" (UniqueName: \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-kube-api-access-lwxg7\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.591144 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.591128    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9c549fd-9a32-4a08-a3ce-bf698639f434-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.591200 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.591168    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca21abb4-1609-4b5e-81b8-72cdc2481417-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.591251 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.591235    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btlmp\" (UniqueName: \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-kube-api-access-btlmp\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.591300 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.591286    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.692170 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692133    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwxg7\" (UniqueName: \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-kube-api-access-lwxg7\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.692380 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692178    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9c549fd-9a32-4a08-a3ce-bf698639f434-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.692380 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692201    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca21abb4-1609-4b5e-81b8-72cdc2481417-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.692380 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692231    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btlmp\" (UniqueName: \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-kube-api-access-btlmp\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.692380 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692292    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.692623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692407    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.692623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692445    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.692623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692473    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.692623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692499    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.692623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692539    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b444a3c-d720-476d-81b9-b4e15407c2e3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.692623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692582    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.692623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692612    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692644    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692686    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692717    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692770    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7twl7\" (UniqueName: \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-kube-api-access-7twl7\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692800    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692840    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692876    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692909    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692933    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692982    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692999    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9c549fd-9a32-4a08-a3ce-bf698639f434-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.693022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693021    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693053    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693089    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693128    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca21abb4-1609-4b5e-81b8-72cdc2481417-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693138    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693188    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693209    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.692978    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693360    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693508    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693623    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693633    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b444a3c-d720-476d-81b9-b4e15407c2e3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.693692 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693635    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.694549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693779    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.694549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693835    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.694549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693880    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.694549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.693915    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.694549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.694125    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.694549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.694359    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.695958 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.695932    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.696238 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.696218    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.696407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.696388    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.696499 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.696481    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.696561 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.696534    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.696617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.696602    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.754756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.754693    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.755363 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.755325    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btlmp\" (UniqueName: \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-kube-api-access-btlmp\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.755495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.755474    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwxg7\" (UniqueName: \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-kube-api-access-lwxg7\") pod \"maas-default-gateway-openshift-default-5bc884dd87-cvpvf\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:43.755495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.755489    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.755720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.755699    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-xfzlb\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.755962 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.755938    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7twl7\" (UniqueName: \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-kube-api-access-7twl7\") pod \"maas-default-gateway-openshift-default-5bc884dd87-mfgcq\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.789901 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.789829    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:43.806184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.806157    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:43.852385 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:43.852352    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:44.240478 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.240443    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"]
Nov 03 16:48:44.244035 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:48:44.243293    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9c549fd_9a32_4a08_a3ce_bf698639f434.slice/crio-d9945a6fde42839ec30c78305c73e34d023169a31af3d92fabb1ffae0eda8e23 WatchSource:0}: Error finding container d9945a6fde42839ec30c78305c73e34d023169a31af3d92fabb1ffae0eda8e23: Status 404 returned error can't find the container with id d9945a6fde42839ec30c78305c73e34d023169a31af3d92fabb1ffae0eda8e23
Nov 03 16:48:44.246790 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.246762    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"]
Nov 03 16:48:44.247658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.247624    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:44.247751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.247692    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:44.247806 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.247720    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:44.247867 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:48:44.247832    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca21abb4_1609_4b5e_81b8_72cdc2481417.slice/crio-f421c7683b6af7034c1b25dbcd03a67eeb4ed5b0e10f1509010930021f3eddd1 WatchSource:0}: Error finding container f421c7683b6af7034c1b25dbcd03a67eeb4ed5b0e10f1509010930021f3eddd1: Status 404 returned error can't find the container with id f421c7683b6af7034c1b25dbcd03a67eeb4ed5b0e10f1509010930021f3eddd1
Nov 03 16:48:44.250420 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.250392    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:44.250535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.250453    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:44.250535 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.250492    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:44.295942 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.295914    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"]
Nov 03 16:48:44.298702 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:48:44.298678    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b444a3c_d720_476d_81b9_b4e15407c2e3.slice/crio-eece10c78f407c9464da2a6b0164a7837ab3a9e2e61e8d02de3cc34a11b75bf0 WatchSource:0}: Error finding container eece10c78f407c9464da2a6b0164a7837ab3a9e2e61e8d02de3cc34a11b75bf0: Status 404 returned error can't find the container with id eece10c78f407c9464da2a6b0164a7837ab3a9e2e61e8d02de3cc34a11b75bf0
Nov 03 16:48:44.301992 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.301957    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:44.302079 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.302039    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:44.302115 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.302078    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:44.567022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.566939    5605 patch_prober.go:29] interesting pod/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:48:44.567022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:44.566999    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:48:45.026753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.025904    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" event={"ID":"1b444a3c-d720-476d-81b9-b4e15407c2e3","Type":"ContainerStarted","Data":"8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0"}
Nov 03 16:48:45.026753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.025981    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" event={"ID":"1b444a3c-d720-476d-81b9-b4e15407c2e3","Type":"ContainerStarted","Data":"eece10c78f407c9464da2a6b0164a7837ab3a9e2e61e8d02de3cc34a11b75bf0"}
Nov 03 16:48:45.048792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.042615    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" event={"ID":"a9c549fd-9a32-4a08-a3ce-bf698639f434","Type":"ContainerStarted","Data":"ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f"}
Nov 03 16:48:45.048792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.042685    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" event={"ID":"a9c549fd-9a32-4a08-a3ce-bf698639f434","Type":"ContainerStarted","Data":"d9945a6fde42839ec30c78305c73e34d023169a31af3d92fabb1ffae0eda8e23"}
Nov 03 16:48:45.064297 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.064250    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" event={"ID":"ca21abb4-1609-4b5e-81b8-72cdc2481417","Type":"ContainerStarted","Data":"40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf"}
Nov 03 16:48:45.064482 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.064323    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" event={"ID":"ca21abb4-1609-4b5e-81b8-72cdc2481417","Type":"ContainerStarted","Data":"f421c7683b6af7034c1b25dbcd03a67eeb4ed5b0e10f1509010930021f3eddd1"}
Nov 03 16:48:45.205757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.182718    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" podStartSLOduration=2.182698265 podStartE2EDuration="2.182698265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:48:45.179519529 +0000 UTC m=+2211.003970799" watchObservedRunningTime="2026-11-03 16:48:45.182698265 +0000 UTC m=+2211.007149540"
Nov 03 16:48:45.324816 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.323901    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" podStartSLOduration=2.323876427 podStartE2EDuration="2.323876427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:48:45.311233974 +0000 UTC m=+2211.135685269" watchObservedRunningTime="2026-11-03 16:48:45.323876427 +0000 UTC m=+2211.148327700"
Nov 03 16:48:45.466462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.466403    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" podStartSLOduration=2.466390311 podStartE2EDuration="2.466390311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:48:45.463990316 +0000 UTC m=+2211.288441588" watchObservedRunningTime="2026-11-03 16:48:45.466390311 +0000 UTC m=+2211.290841657"
Nov 03 16:48:45.793752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.793168    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:45.798205 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.798169    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-xfzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 03 16:48:45.798483 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.798454    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 03 16:48:45.810679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.810646    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:45.811034 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.811010    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-mfgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Nov 03 16:48:45.811257 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.811229    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Nov 03 16:48:45.860493 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.860115    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:45.860892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.860863    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-cvpvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Nov 03 16:48:45.861063 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:45.861040    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Nov 03 16:48:46.813761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:46.810511    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-xfzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 03 16:48:46.860897 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:46.860855    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 03 16:48:46.865093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:46.833954    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-mfgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Nov 03 16:48:46.865093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:46.864226    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Nov 03 16:48:46.865093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:46.864600    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-cvpvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Nov 03 16:48:46.865093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:46.864642    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Nov 03 16:48:47.801756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:47.798160    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-xfzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 03 16:48:47.801756 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:47.798224    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 03 16:48:47.812183 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:47.812146    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-mfgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Nov 03 16:48:47.812445 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:47.812418    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Nov 03 16:48:47.855395 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:47.855362    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-cvpvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Nov 03 16:48:47.856611 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:47.856579    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Nov 03 16:48:48.823751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:48.823400    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-mfgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Nov 03 16:48:48.823751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:48.823458    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Nov 03 16:48:48.824006 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:48.823891    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-xfzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 03 16:48:48.824006 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:48.823929    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 03 16:48:48.858966 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:48.858935    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-cvpvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Nov 03 16:48:48.859568 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:48.859537    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Nov 03 16:48:48.976373 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:48.976265    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"]
Nov 03 16:48:48.976557 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:48.976509    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy" containerID="cri-o://049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085" gracePeriod=30
Nov 03 16:48:48.990948 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:48.990916    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:48:49.085154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.085123    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-token\" (UniqueName: \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-token\") pod \"423c492d-e0e6-442f-949d-c4cee39a9d04\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") "
Nov 03 16:48:49.085793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.085769    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-envoy\") pod \"423c492d-e0e6-442f-949d-c4cee39a9d04\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") "
Nov 03 16:48:49.085992 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.085977    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-certs\") pod \"423c492d-e0e6-442f-949d-c4cee39a9d04\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") "
Nov 03 16:48:49.086114 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.086102    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-credential-socket\") pod \"423c492d-e0e6-442f-949d-c4cee39a9d04\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") "
Nov 03 16:48:49.086220 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.086210    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/423c492d-e0e6-442f-949d-c4cee39a9d04-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/423c492d-e0e6-442f-949d-c4cee39a9d04-istiod-ca-cert\") pod \"423c492d-e0e6-442f-949d-c4cee39a9d04\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") "
Nov 03 16:48:49.086382 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.086370    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-data\") pod \"423c492d-e0e6-442f-949d-c4cee39a9d04\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") "
Nov 03 16:48:49.086542 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.086529    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-socket\") pod \"423c492d-e0e6-442f-949d-c4cee39a9d04\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") "
Nov 03 16:48:49.087720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.087418    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-certs" pod "423c492d-e0e6-442f-949d-c4cee39a9d04" (UID: "423c492d-e0e6-442f-949d-c4cee39a9d04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:49.087720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.087489    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-data" pod "423c492d-e0e6-442f-949d-c4cee39a9d04" (UID: "423c492d-e0e6-442f-949d-c4cee39a9d04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:49.087720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.087690    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-socket" pod "423c492d-e0e6-442f-949d-c4cee39a9d04" (UID: "423c492d-e0e6-442f-949d-c4cee39a9d04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:49.088269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.088245    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/423c492d-e0e6-442f-949d-c4cee39a9d04-istiod-ca-cert" pod "423c492d-e0e6-442f-949d-c4cee39a9d04" (UID: "423c492d-e0e6-442f-949d-c4cee39a9d04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:48:49.089939 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.089917    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-credential-socket" pod "423c492d-e0e6-442f-949d-c4cee39a9d04" (UID: "423c492d-e0e6-442f-949d-c4cee39a9d04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:49.090132 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.090062    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-kube-api-access-stwl9\" (UniqueName: \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-kube-api-access-stwl9\") pod \"423c492d-e0e6-442f-949d-c4cee39a9d04\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") "
Nov 03 16:48:49.090674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.090656    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-podinfo\") pod \"423c492d-e0e6-442f-949d-c4cee39a9d04\" (UID: \"423c492d-e0e6-442f-949d-c4cee39a9d04\") "
Nov 03 16:48:49.091548 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.091527    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:49.097953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.097931    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:49.098280 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.098261    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:49.098398 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.098383    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:49.098481 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.098470    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/423c492d-e0e6-442f-949d-c4cee39a9d04-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:49.269800 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.263963    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-envoy" pod "423c492d-e0e6-442f-949d-c4cee39a9d04" (UID: "423c492d-e0e6-442f-949d-c4cee39a9d04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:49.271751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.271038    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-token" pod "423c492d-e0e6-442f-949d-c4cee39a9d04" (UID: "423c492d-e0e6-442f-949d-c4cee39a9d04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:48:49.271751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.271632    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-podinfo" pod "423c492d-e0e6-442f-949d-c4cee39a9d04" (UID: "423c492d-e0e6-442f-949d-c4cee39a9d04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:48:49.272108 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.272080    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-kube-api-access-stwl9" pod "423c492d-e0e6-442f-949d-c4cee39a9d04" (UID: "423c492d-e0e6-442f-949d-c4cee39a9d04"). InnerVolumeSpecName "kube-api-access-stwl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:48:49.274810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.274785    5605 generic.go:356] "Generic (PLEG): container finished" podID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerID="f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb" exitCode=0
Nov 03 16:48:49.274895 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.274838    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"
Nov 03 16:48:49.274895 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.274858    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" event={"ID":"423c492d-e0e6-442f-949d-c4cee39a9d04","Type":"ContainerDied","Data":"f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb"}
Nov 03 16:48:49.274971 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.274911    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2" event={"ID":"423c492d-e0e6-442f-949d-c4cee39a9d04","Type":"ContainerDied","Data":"2d24653961b571d899c4325e37a6b86d1d412864d6fbb8d867e430d3e5b20b27"}
Nov 03 16:48:49.274971 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.274937    5605 scope.go:122] "RemoveContainer" containerID="f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb"
Nov 03 16:48:49.374887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.356432    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stwl9\" (UniqueName: \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-kube-api-access-stwl9\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:49.374887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.356469    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:49.374887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.356485    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:49.374887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.356500    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/423c492d-e0e6-442f-949d-c4cee39a9d04-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:49.374887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.371374    5605 scope.go:122] "RemoveContainer" containerID="fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8"
Nov 03 16:48:49.479785 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.479761    5605 scope.go:122] "RemoveContainer" containerID="f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb"
Nov 03 16:48:49.480377 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:48:49.480349    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb\": container with ID starting with f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb not found: ID does not exist" containerID="f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb"
Nov 03 16:48:49.480559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.480388    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb"} err="failed to get container status \"f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb\": rpc error: code = NotFound desc = could not find container \"f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb\": container with ID starting with f0ccf957b977f18291f3ba82c06b053cd12fdb05ed3742e28679850cf77d26cb not found: ID does not exist"
Nov 03 16:48:49.480559 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.480410    5605 scope.go:122] "RemoveContainer" containerID="fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8"
Nov 03 16:48:49.506753 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:48:49.482269    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8\": container with ID starting with fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8 not found: ID does not exist" containerID="fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8"
Nov 03 16:48:49.507270 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.507231    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8"} err="failed to get container status \"fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8\": rpc error: code = NotFound desc = could not find container \"fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8\": container with ID starting with fc97dc3022dbe4802cb0f095bd823aef33601f0d730d1f318bf96307adab52b8 not found: ID does not exist"
Nov 03 16:48:49.562835 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.562301    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"]
Nov 03 16:48:49.565950 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.565694    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-cc4dcd-openshift-default-5c76c44f59-8z2d2"]
Nov 03 16:48:49.799753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.792134    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-xfzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 03 16:48:49.799753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.792195    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 03 16:48:49.821812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.808503    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-mfgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Nov 03 16:48:49.821812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.808563    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Nov 03 16:48:49.854223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.853807    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-cvpvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Nov 03 16:48:49.854223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:49.853871    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Nov 03 16:48:50.365848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:50.365569    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" path="/var/lib/kubelet/pods/423c492d-e0e6-442f-949d-c4cee39a9d04/volumes"
Nov 03 16:48:50.798812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:50.798130    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-xfzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 03 16:48:50.798812 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:50.798196    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 03 16:48:50.808569 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:50.807335    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-mfgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Nov 03 16:48:50.808569 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:50.807393    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Nov 03 16:48:50.853123 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:50.853079    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-cvpvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Nov 03 16:48:50.853301 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:50.853139    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Nov 03 16:48:51.598529 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.598496    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"]
Nov 03 16:48:51.599431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.599406    5605 cpu_manager.go:414] "containerMap: removing container" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy"
Nov 03 16:48:51.599560 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.599435    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy"
Nov 03 16:48:51.599642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.599626    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy"
Nov 03 16:48:51.605229 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.605195    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.687926 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.687891    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3b4d9b-openshift-default-dockercfg-djj95\""
Nov 03 16:48:51.815753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.793063    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"]
Nov 03 16:48:51.816168 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.796426    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-xfzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Nov 03 16:48:51.840757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.821865    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Nov 03 16:48:51.841073 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.798167    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-envoy\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.841362 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.841336    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-data\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.841539 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.841515    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-certs\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.845750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.841703    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-socket\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.853793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.849990    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-token\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.866927 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.866886    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istiod-ca-cert\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.868550 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.868526    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-podinfo\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.874450 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.874424    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-credential-socket\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.874859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.874836    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thg2t\" (UniqueName: \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-kube-api-access-thg2t\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.875217 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.824466    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-mfgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Nov 03 16:48:51.875384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.875354    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Nov 03 16:48:51.875579 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.868377    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-cvpvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Nov 03 16:48:51.875757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.875709    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Nov 03 16:48:51.976488 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.976447    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istiod-ca-cert\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.976681 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.976508    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-podinfo\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.976681 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.976542    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-credential-socket\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.976681 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.976576    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thg2t\" (UniqueName: \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-kube-api-access-thg2t\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.976921 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.976686    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-envoy\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.976921 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.976829    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-data\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.976921 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.976876    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-certs\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.977078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.976948    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-socket\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.977078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.976989    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-token\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.977702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.977610    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-data\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.977702 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.977650    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-certs\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.978194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.978043    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-socket\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.978194 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.978104    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-credential-socket\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.978595 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.978565    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istiod-ca-cert\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.980923 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.980894    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-envoy\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:51.981403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:51.981379    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-podinfo\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:52.038850 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:52.038809    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thg2t\" (UniqueName: \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-kube-api-access-thg2t\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:52.041597 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:52.041566    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-token\") pod \"e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") " pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:52.045426 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:52.045399    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:52.574938 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:52.574847    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"]
Nov 03 16:48:52.593308 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:48:52.593274    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd0aeaa_25c4_49f1_8758_bbccf9f16b96.slice/crio-75c02b88c0d224848728ffd1a51a935eeca469d6aed55374d281dae25c5b81dd WatchSource:0}: Error finding container 75c02b88c0d224848728ffd1a51a935eeca469d6aed55374d281dae25c5b81dd: Status 404 returned error can't find the container with id 75c02b88c0d224848728ffd1a51a935eeca469d6aed55374d281dae25c5b81dd
Nov 03 16:48:52.596928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:52.596885    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:52.597047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:52.596973    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:52.597047 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:52.597032    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:48:53.341506 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:53.340381    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" event={"ID":"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96","Type":"ContainerStarted","Data":"af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83"}
Nov 03 16:48:53.341506 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:53.340453    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" event={"ID":"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96","Type":"ContainerStarted","Data":"75c02b88c0d224848728ffd1a51a935eeca469d6aed55374d281dae25c5b81dd"}
Nov 03 16:48:53.832969 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:53.832868    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-xfzlb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:48:53.832969 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:53.832941    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:48:53.849924 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:53.849884    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-mfgcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:48:53.850106 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:53.849978    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:48:53.853791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:53.853763    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-cvpvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:48:53.854030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:53.853998    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:48:54.019439 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.019407    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:54.021293 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.021263    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:54.024380 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.024337    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:48:54.029024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.028958    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:54.030124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.030040    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:54.031591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.031570    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:48:54.045781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.045690    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:54.052213 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.052185    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:54.060767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.059561    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:54.060767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.060568    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:54.061584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.061394    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:48:54.148067 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.148015    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" podStartSLOduration=3.148000171 podStartE2EDuration="3.148000171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:48:53.472501971 +0000 UTC m=+2219.296953255" watchObservedRunningTime="2026-11-03 16:48:54.148000171 +0000 UTC m=+2219.972451442"
Nov 03 16:48:54.271322 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.271290    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:48:54.352233 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.352198    5605 generic.go:356] "Generic (PLEG): container finished" podID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerID="049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085" exitCode=0
Nov 03 16:48:54.352703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.352248    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"
Nov 03 16:48:54.352703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.352278    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" event={"ID":"8114cb0a-6d4e-4b87-b562-03cf2fec6319","Type":"ContainerDied","Data":"049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085"}
Nov 03 16:48:54.352703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.352341    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" event={"ID":"8114cb0a-6d4e-4b87-b562-03cf2fec6319","Type":"ContainerDied","Data":"a4678da25a569411300f2d05dec243611ce01ab1740a643edd69defd2172dab3"}
Nov 03 16:48:54.352703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.352376    5605 scope.go:122] "RemoveContainer" containerID="049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085"
Nov 03 16:48:54.352703 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.352640    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:54.353858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.353829    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:48:54.362686 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.362669    5605 scope.go:122] "RemoveContainer" containerID="73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531"
Nov 03 16:48:54.371780 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.371763    5605 scope.go:122] "RemoveContainer" containerID="049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085"
Nov 03 16:48:54.372076 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:48:54.372055    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085\": container with ID starting with 049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085 not found: ID does not exist" containerID="049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085"
Nov 03 16:48:54.372157 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.372084    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085"} err="failed to get container status \"049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085\": rpc error: code = NotFound desc = could not find container \"049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085\": container with ID starting with 049478a73ce1c27a353922ff4feba273d9f7fe0440666f0f6264927707060085 not found: ID does not exist"
Nov 03 16:48:54.372157 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.372103    5605 scope.go:122] "RemoveContainer" containerID="73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531"
Nov 03 16:48:54.372355 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:48:54.372339    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531\": container with ID starting with 73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531 not found: ID does not exist" containerID="73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531"
Nov 03 16:48:54.372406 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.372360    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531"} err="failed to get container status \"73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531\": rpc error: code = NotFound desc = could not find container \"73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531\": container with ID starting with 73db04d722f3377c24f284d839a2e55c7adf15724451be49cc61be7552059531 not found: ID does not exist"
Nov 03 16:48:54.428086 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428058    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-certs\") pod \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") "
Nov 03 16:48:54.428269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428102    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-token\" (UniqueName: \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-token\") pod \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") "
Nov 03 16:48:54.428269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428131    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-podinfo\") pod \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") "
Nov 03 16:48:54.428269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428193    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-credential-socket\") pod \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") "
Nov 03 16:48:54.428442 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428329    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-certs" pod "8114cb0a-6d4e-4b87-b562-03cf2fec6319" (UID: "8114cb0a-6d4e-4b87-b562-03cf2fec6319"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:54.428442 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428338    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-data\") pod \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") "
Nov 03 16:48:54.428556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428497    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-envoy\") pod \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") "
Nov 03 16:48:54.428556 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428501    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-credential-socket" pod "8114cb0a-6d4e-4b87-b562-03cf2fec6319" (UID: "8114cb0a-6d4e-4b87-b562-03cf2fec6319"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:54.428679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428553    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-data" pod "8114cb0a-6d4e-4b87-b562-03cf2fec6319" (UID: "8114cb0a-6d4e-4b87-b562-03cf2fec6319"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:54.428679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428579    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istiod-ca-cert\") pod \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") "
Nov 03 16:48:54.428679 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428643    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-socket\") pod \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") "
Nov 03 16:48:54.428867 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428698    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-kube-api-access-jt25k\" (UniqueName: \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-kube-api-access-jt25k\") pod \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\" (UID: \"8114cb0a-6d4e-4b87-b562-03cf2fec6319\") "
Nov 03 16:48:54.428953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.428929    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-socket" pod "8114cb0a-6d4e-4b87-b562-03cf2fec6319" (UID: "8114cb0a-6d4e-4b87-b562-03cf2fec6319"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:54.429184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.429155    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istiod-ca-cert" pod "8114cb0a-6d4e-4b87-b562-03cf2fec6319" (UID: "8114cb0a-6d4e-4b87-b562-03cf2fec6319"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:48:54.430267 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.430242    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:54.430409 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.430272    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:54.430409 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.430299    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:54.430409 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.430319    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:54.430409 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.430329    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:54.430828 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.430799    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-token" pod "8114cb0a-6d4e-4b87-b562-03cf2fec6319" (UID: "8114cb0a-6d4e-4b87-b562-03cf2fec6319"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:48:54.430902 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.430874    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-podinfo" pod "8114cb0a-6d4e-4b87-b562-03cf2fec6319" (UID: "8114cb0a-6d4e-4b87-b562-03cf2fec6319"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:48:54.431567 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.431549    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-envoy" pod "8114cb0a-6d4e-4b87-b562-03cf2fec6319" (UID: "8114cb0a-6d4e-4b87-b562-03cf2fec6319"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:48:54.431617 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.431568    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-kube-api-access-jt25k" pod "8114cb0a-6d4e-4b87-b562-03cf2fec6319" (UID: "8114cb0a-6d4e-4b87-b562-03cf2fec6319"). InnerVolumeSpecName "kube-api-access-jt25k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:48:54.531657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.531622    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:54.531657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.531653    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt25k\" (UniqueName: \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-kube-api-access-jt25k\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:54.531657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.531665    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:54.531995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:54.531674    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8114cb0a-6d4e-4b87-b562-03cf2fec6319-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:48:55.074516 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:55.074428    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"]
Nov 03 16:48:55.081479 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:55.081451    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5"]
Nov 03 16:48:55.256276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:55.256242    5605 patch_prober.go:29] interesting pod/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 16:48:55.256446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:55.256299    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-6f58cf-openshift-default-5fd846bf9c-ncdm5" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:48:56.323230 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:48:56.322774    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" path="/var/lib/kubelet/pods/8114cb0a-6d4e-4b87-b562-03cf2fec6319/volumes"
Nov 03 16:49:00.250794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.250720    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"]
Nov 03 16:49:00.251596 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.251576    5605 cpu_manager.go:414] "containerMap: removing container" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy"
Nov 03 16:49:00.251648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.251598    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy"
Nov 03 16:49:00.251648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.251635    5605 cpu_manager.go:414] "containerMap: removing container" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy"
Nov 03 16:49:00.251648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.251640    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy"
Nov 03 16:49:00.251767 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.251760    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy"
Nov 03 16:49:00.251804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.251770    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="423c492d-e0e6-442f-949d-c4cee39a9d04" containerName="istio-proxy"
Nov 03 16:49:00.257290 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.257228    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:49:00.312797 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.312759    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"]
Nov 03 16:49:00.327148 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.327113    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29ac9424-33f5-48f0-83e2-022d1b036959-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x\" (UID: \"29ac9424-33f5-48f0-83e2-022d1b036959\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:49:00.327377 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.327357    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2w2r\" (UniqueName: \"kubernetes.io/projected/29ac9424-33f5-48f0-83e2-022d1b036959-kube-api-access-z2w2r\") pod \"payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x\" (UID: \"29ac9424-33f5-48f0-83e2-022d1b036959\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:49:00.359195 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.359161    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-3b4d9b\""
Nov 03 16:49:00.359362 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.359318    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-3b4d9b-dockercfg-8f8lv\""
Nov 03 16:49:00.439749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.430663    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2w2r\" (UniqueName: \"kubernetes.io/projected/29ac9424-33f5-48f0-83e2-022d1b036959-kube-api-access-z2w2r\") pod \"payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x\" (UID: \"29ac9424-33f5-48f0-83e2-022d1b036959\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:49:00.439749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.430898    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29ac9424-33f5-48f0-83e2-022d1b036959-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x\" (UID: \"29ac9424-33f5-48f0-83e2-022d1b036959\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:49:00.439749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.431937    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29ac9424-33f5-48f0-83e2-022d1b036959-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x\" (UID: \"29ac9424-33f5-48f0-83e2-022d1b036959\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:49:00.477858 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.460600    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"]
Nov 03 16:49:00.487701 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.487665    5605 cpu_manager.go:414] "containerMap: removing container" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy"
Nov 03 16:49:00.487943 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.487926    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy"
Nov 03 16:49:00.488283 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.488266    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="8114cb0a-6d4e-4b87-b562-03cf2fec6319" containerName="istio-proxy"
Nov 03 16:49:00.494519 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.494497    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:49:00.499242 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.498542    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"]
Nov 03 16:49:00.502889 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.502832    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2w2r\" (UniqueName: \"kubernetes.io/projected/29ac9424-33f5-48f0-83e2-022d1b036959-kube-api-access-z2w2r\") pod \"payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x\" (UID: \"29ac9424-33f5-48f0-83e2-022d1b036959\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:49:00.533279 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.533247    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn6lw\" (UniqueName: \"kubernetes.io/projected/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-kube-api-access-cn6lw\") pod \"payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd\" (UID: \"f5c1f5f4-78bb-4305-9213-1660fcc23cbf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:49:00.533446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.533389    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd\" (UID: \"f5c1f5f4-78bb-4305-9213-1660fcc23cbf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:49:00.572089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.572063    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:49:00.635143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.634691    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn6lw\" (UniqueName: \"kubernetes.io/projected/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-kube-api-access-cn6lw\") pod \"payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd\" (UID: \"f5c1f5f4-78bb-4305-9213-1660fcc23cbf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:49:00.635143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.634902    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd\" (UID: \"f5c1f5f4-78bb-4305-9213-1660fcc23cbf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:49:00.635942 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.635821    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd\" (UID: \"f5c1f5f4-78bb-4305-9213-1660fcc23cbf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:49:00.696423 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.696387    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn6lw\" (UniqueName: \"kubernetes.io/projected/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-kube-api-access-cn6lw\") pod \"payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd\" (UID: \"f5c1f5f4-78bb-4305-9213-1660fcc23cbf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:49:00.829101 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.829024    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:49:00.956572 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:49:00.956525    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ac9424_33f5_48f0_83e2_022d1b036959.slice/crio-a9fbbc30330182f3a219a581a0a537694e4aa40251567d206c9378e9dfc685bf WatchSource:0}: Error finding container a9fbbc30330182f3a219a581a0a537694e4aa40251567d206c9378e9dfc685bf: Status 404 returned error can't find the container with id a9fbbc30330182f3a219a581a0a537694e4aa40251567d206c9378e9dfc685bf
Nov 03 16:49:00.958525 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:00.958494    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"]
Nov 03 16:49:01.212129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:01.212102    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"]
Nov 03 16:49:01.212983 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:49:01.212953    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5c1f5f4_78bb_4305_9213_1660fcc23cbf.slice/crio-76feefd353834a18702d312f7a1762c829528f144c9507fa9c70dabacd3310e8 WatchSource:0}: Error finding container 76feefd353834a18702d312f7a1762c829528f144c9507fa9c70dabacd3310e8: Status 404 returned error can't find the container with id 76feefd353834a18702d312f7a1762c829528f144c9507fa9c70dabacd3310e8
Nov 03 16:49:01.468707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:01.468599    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x" event={"ID":"29ac9424-33f5-48f0-83e2-022d1b036959","Type":"ContainerStarted","Data":"3e4b847b430db08147c4917c7e1e598a42c316ea688dc1f30f78a7b34854bbd3"}
Nov 03 16:49:01.468707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:01.468671    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:49:01.468707 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:01.468695    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x" event={"ID":"29ac9424-33f5-48f0-83e2-022d1b036959","Type":"ContainerStarted","Data":"a9fbbc30330182f3a219a581a0a537694e4aa40251567d206c9378e9dfc685bf"}
Nov 03 16:49:01.470317 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:01.470293    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd" event={"ID":"f5c1f5f4-78bb-4305-9213-1660fcc23cbf","Type":"ContainerStarted","Data":"bc0fe906f028f0ea58e437d60307b606047582e96bba0d8b37ffc80b91a2097a"}
Nov 03 16:49:01.470448 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:01.470342    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:49:01.470448 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:01.470360    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd" event={"ID":"f5c1f5f4-78bb-4305-9213-1660fcc23cbf","Type":"ContainerStarted","Data":"76feefd353834a18702d312f7a1762c829528f144c9507fa9c70dabacd3310e8"}
Nov 03 16:49:01.592878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:01.592826    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x" podStartSLOduration=1.592812041 podStartE2EDuration="1.592812041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:49:01.591239385 +0000 UTC m=+2227.415690659" watchObservedRunningTime="2026-11-03 16:49:01.592812041 +0000 UTC m=+2227.417263382"
Nov 03 16:49:01.733118 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:01.732988    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd" podStartSLOduration=1.7329691459999998 podStartE2EDuration="1.732969146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:49:01.730305483 +0000 UTC m=+2227.554756774" watchObservedRunningTime="2026-11-03 16:49:01.732969146 +0000 UTC m=+2227.557420418"
Nov 03 16:49:03.394836 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:49:03.394796    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:49:09.825962 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:09.825925    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"]
Nov 03 16:49:09.826387 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:09.826118    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd" podUID="d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb" containerName="maas-api" containerID="cri-o://00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94" gracePeriod=30
Nov 03 16:49:10.136813 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.136782    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:49:10.273752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.273154    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-openshift-service-ca\") pod \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") "
Nov 03 16:49:10.273752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.273277    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-metrics-certs\") pod \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") "
Nov 03 16:49:10.273752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.273305    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-maas-api-tls\") pod \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") "
Nov 03 16:49:10.273752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.273372    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-kube-api-access-4srx5\" (UniqueName: \"kubernetes.io/projected/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-kube-api-access-4srx5\") pod \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\" (UID: \"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb\") "
Nov 03 16:49:10.284761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.284292    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-kube-api-access-4srx5" pod "d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb" (UID: "d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb"). InnerVolumeSpecName "kube-api-access-4srx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:49:10.306830 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.304827    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-openshift-service-ca" pod "d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb" (UID: "d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:49:10.323989 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.323955    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-maas-api-tls" pod "d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb" (UID: "d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:49:10.327412 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.327359    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-metrics-certs" pod "d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb" (UID: "d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:49:10.381809 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.377831    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:10.381809 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.377865    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:10.381809 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.377881    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:10.381809 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.377896    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4srx5\" (UniqueName: \"kubernetes.io/projected/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb-kube-api-access-4srx5\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:10.526165 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.526134    5605 generic.go:356] "Generic (PLEG): container finished" podID="d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb" containerID="00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94" exitCode=0
Nov 03 16:49:10.526343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.526181    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"
Nov 03 16:49:10.526343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.526211    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd" event={"ID":"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb","Type":"ContainerDied","Data":"00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94"}
Nov 03 16:49:10.526343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.526265    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd" event={"ID":"d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb","Type":"ContainerDied","Data":"3268bdbf5dbd47c69bd0bb6dd3ddc787ab18905b6fa7d9559be2ac2d2327e78a"}
Nov 03 16:49:10.526343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.526286    5605 scope.go:122] "RemoveContainer" containerID="00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94"
Nov 03 16:49:10.536622 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.536604    5605 scope.go:122] "RemoveContainer" containerID="00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94"
Nov 03 16:49:10.536938 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:49:10.536915    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94\": container with ID starting with 00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94 not found: ID does not exist" containerID="00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94"
Nov 03 16:49:10.537042 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.536944    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94"} err="failed to get container status \"00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94\": rpc error: code = NotFound desc = could not find container \"00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94\": container with ID starting with 00d5a457d8e1dbdb1ab17433ce317b331cb6ec6680ede7f198229149412e8c94 not found: ID does not exist"
Nov 03 16:49:10.699501 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.699412    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"]
Nov 03 16:49:10.705613 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:10.705587    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cb494a-787bc6cf68-wrbdd"]
Nov 03 16:49:12.309078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:12.309042    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb" path="/var/lib/kubelet/pods/d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb/volumes"
Nov 03 16:49:12.479825 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:12.479796    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:49:12.480025 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:12.479874    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:49:20.261265 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:20.261227    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"]
Nov 03 16:49:20.261697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:20.261437    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy" containerID="cri-o://d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18" gracePeriod=30
Nov 03 16:49:25.538358 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.538294    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:49:25.594031 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.593997    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-credential-socket\") pod \"afb0f028-ecde-48ed-92b0-39276f7306e7\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") "
Nov 03 16:49:25.594201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594045    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-podinfo\") pod \"afb0f028-ecde-48ed-92b0-39276f7306e7\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") "
Nov 03 16:49:25.594201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594082    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-token\" (UniqueName: \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-token\") pod \"afb0f028-ecde-48ed-92b0-39276f7306e7\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") "
Nov 03 16:49:25.594201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594114    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-certs\") pod \"afb0f028-ecde-48ed-92b0-39276f7306e7\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") "
Nov 03 16:49:25.594201 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594172    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/afb0f028-ecde-48ed-92b0-39276f7306e7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afb0f028-ecde-48ed-92b0-39276f7306e7-istiod-ca-cert\") pod \"afb0f028-ecde-48ed-92b0-39276f7306e7\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") "
Nov 03 16:49:25.594407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594268    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-data\") pod \"afb0f028-ecde-48ed-92b0-39276f7306e7\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") "
Nov 03 16:49:25.594407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594302    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-kube-api-access-dt99k\" (UniqueName: \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-kube-api-access-dt99k\") pod \"afb0f028-ecde-48ed-92b0-39276f7306e7\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") "
Nov 03 16:49:25.594407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594359    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-envoy\") pod \"afb0f028-ecde-48ed-92b0-39276f7306e7\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") "
Nov 03 16:49:25.594407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594395    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-socket\") pod \"afb0f028-ecde-48ed-92b0-39276f7306e7\" (UID: \"afb0f028-ecde-48ed-92b0-39276f7306e7\") "
Nov 03 16:49:25.594622 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594295    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-credential-socket" pod "afb0f028-ecde-48ed-92b0-39276f7306e7" (UID: "afb0f028-ecde-48ed-92b0-39276f7306e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:25.594622 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594426    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-certs" pod "afb0f028-ecde-48ed-92b0-39276f7306e7" (UID: "afb0f028-ecde-48ed-92b0-39276f7306e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:25.594622 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594512    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-data" pod "afb0f028-ecde-48ed-92b0-39276f7306e7" (UID: "afb0f028-ecde-48ed-92b0-39276f7306e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:25.594847 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594819    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-socket" pod "afb0f028-ecde-48ed-92b0-39276f7306e7" (UID: "afb0f028-ecde-48ed-92b0-39276f7306e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:25.594949 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.594917    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afb0f028-ecde-48ed-92b0-39276f7306e7-istiod-ca-cert" pod "afb0f028-ecde-48ed-92b0-39276f7306e7" (UID: "afb0f028-ecde-48ed-92b0-39276f7306e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:49:25.595260 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.595218    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:25.595260 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.595241    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afb0f028-ecde-48ed-92b0-39276f7306e7-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:25.595260 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.595256    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:25.595462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.595270    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:25.595462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.595283    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:25.596657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.596632    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-token" pod "afb0f028-ecde-48ed-92b0-39276f7306e7" (UID: "afb0f028-ecde-48ed-92b0-39276f7306e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:49:25.596816 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.596635    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-podinfo" pod "afb0f028-ecde-48ed-92b0-39276f7306e7" (UID: "afb0f028-ecde-48ed-92b0-39276f7306e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:49:25.596816 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.596704    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-envoy" pod "afb0f028-ecde-48ed-92b0-39276f7306e7" (UID: "afb0f028-ecde-48ed-92b0-39276f7306e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:25.597140 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.597120    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-kube-api-access-dt99k" pod "afb0f028-ecde-48ed-92b0-39276f7306e7" (UID: "afb0f028-ecde-48ed-92b0-39276f7306e7"). InnerVolumeSpecName "kube-api-access-dt99k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:49:25.619848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.619824    5605 generic.go:356] "Generic (PLEG): container finished" podID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerID="d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18" exitCode=0
Nov 03 16:49:25.619957 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.619887    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" event={"ID":"afb0f028-ecde-48ed-92b0-39276f7306e7","Type":"ContainerDied","Data":"d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18"}
Nov 03 16:49:25.619957 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.619940    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q" event={"ID":"afb0f028-ecde-48ed-92b0-39276f7306e7","Type":"ContainerDied","Data":"000ca327e07cccef2bd355040e490312ba84764bf8e416c0da425a108c6c7df8"}
Nov 03 16:49:25.619957 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.619944    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"
Nov 03 16:49:25.620083 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.619962    5605 scope.go:122] "RemoveContainer" containerID="d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18"
Nov 03 16:49:25.630509 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.630489    5605 scope.go:122] "RemoveContainer" containerID="94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981"
Nov 03 16:49:25.639426 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.639410    5605 scope.go:122] "RemoveContainer" containerID="d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18"
Nov 03 16:49:25.639705 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:49:25.639683    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18\": container with ID starting with d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18 not found: ID does not exist" containerID="d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18"
Nov 03 16:49:25.639806 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.639713    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18"} err="failed to get container status \"d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18\": rpc error: code = NotFound desc = could not find container \"d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18\": container with ID starting with d79a0a509067ed74445cb09e34e0a618c99720156f1e07a5a47682f016e36a18 not found: ID does not exist"
Nov 03 16:49:25.639806 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.639751    5605 scope.go:122] "RemoveContainer" containerID="94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981"
Nov 03 16:49:25.640021 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:49:25.639999    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981\": container with ID starting with 94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981 not found: ID does not exist" containerID="94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981"
Nov 03 16:49:25.640080 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.640032    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981"} err="failed to get container status \"94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981\": rpc error: code = NotFound desc = could not find container \"94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981\": container with ID starting with 94c3febea6e36575cec84d25fc08a3a99e7f6f206e60b3c6ffdb0dad8bf2b981 not found: ID does not exist"
Nov 03 16:49:25.696341 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.696312    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:25.696341 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.696338    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:25.696590 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.696351    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt99k\" (UniqueName: \"kubernetes.io/projected/afb0f028-ecde-48ed-92b0-39276f7306e7-kube-api-access-dt99k\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:25.696590 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.696361    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afb0f028-ecde-48ed-92b0-39276f7306e7-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:25.830352 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.830265    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"]
Nov 03 16:49:25.836011 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:25.835983    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-cb494a-openshift-default-5664bdfb4c-t2h2q"]
Nov 03 16:49:26.270407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.270373    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"]
Nov 03 16:49:26.270648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.270619    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm" podUID="24c2b816-5bf9-40f9-98d4-e325d16a869c" containerName="maas-api" containerID="cri-o://c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0" gracePeriod=30
Nov 03 16:49:26.316549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.316511    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" path="/var/lib/kubelet/pods/afb0f028-ecde-48ed-92b0-39276f7306e7/volumes"
Nov 03 16:49:26.587443 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.587419    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:49:26.627859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.627828    5605 generic.go:356] "Generic (PLEG): container finished" podID="24c2b816-5bf9-40f9-98d4-e325d16a869c" containerID="c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0" exitCode=0
Nov 03 16:49:26.628015 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.627866    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm" event={"ID":"24c2b816-5bf9-40f9-98d4-e325d16a869c","Type":"ContainerDied","Data":"c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0"}
Nov 03 16:49:26.628015 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.627875    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"
Nov 03 16:49:26.628015 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.627934    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm" event={"ID":"24c2b816-5bf9-40f9-98d4-e325d16a869c","Type":"ContainerDied","Data":"98e68074a3e7543a8b8de3904782001b5da49106deff7183548af1dced917ae3"}
Nov 03 16:49:26.628015 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.627964    5605 scope.go:122] "RemoveContainer" containerID="c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0"
Nov 03 16:49:26.640646 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.640613    5605 scope.go:122] "RemoveContainer" containerID="c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0"
Nov 03 16:49:26.640983 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:49:26.640965    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0\": container with ID starting with c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0 not found: ID does not exist" containerID="c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0"
Nov 03 16:49:26.641065 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.640992    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0"} err="failed to get container status \"c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0\": rpc error: code = NotFound desc = could not find container \"c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0\": container with ID starting with c91cf49affb1046079252ee5a6d79ed13b98c96814ca3bef3f4b84fcfc7125a0 not found: ID does not exist"
Nov 03 16:49:26.744453 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.734197    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-maas-api-tls\") pod \"24c2b816-5bf9-40f9-98d4-e325d16a869c\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") "
Nov 03 16:49:26.744453 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.742300    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24c2b816-5bf9-40f9-98d4-e325d16a869c-kube-api-access-kpp5v\" (UniqueName: \"kubernetes.io/projected/24c2b816-5bf9-40f9-98d4-e325d16a869c-kube-api-access-kpp5v\") pod \"24c2b816-5bf9-40f9-98d4-e325d16a869c\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") "
Nov 03 16:49:26.761935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.748121    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24c2b816-5bf9-40f9-98d4-e325d16a869c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c2b816-5bf9-40f9-98d4-e325d16a869c-openshift-service-ca\") pod \"24c2b816-5bf9-40f9-98d4-e325d16a869c\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") "
Nov 03 16:49:26.761935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.748210    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-metrics-certs\") pod \"24c2b816-5bf9-40f9-98d4-e325d16a869c\" (UID: \"24c2b816-5bf9-40f9-98d4-e325d16a869c\") "
Nov 03 16:49:26.761935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.752195    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c2b816-5bf9-40f9-98d4-e325d16a869c-openshift-service-ca" pod "24c2b816-5bf9-40f9-98d4-e325d16a869c" (UID: "24c2b816-5bf9-40f9-98d4-e325d16a869c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:49:26.771966 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.768908    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-maas-api-tls" pod "24c2b816-5bf9-40f9-98d4-e325d16a869c" (UID: "24c2b816-5bf9-40f9-98d4-e325d16a869c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:49:26.788627 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.788581    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-metrics-certs" pod "24c2b816-5bf9-40f9-98d4-e325d16a869c" (UID: "24c2b816-5bf9-40f9-98d4-e325d16a869c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:49:26.791260 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.791007    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c2b816-5bf9-40f9-98d4-e325d16a869c-kube-api-access-kpp5v" pod "24c2b816-5bf9-40f9-98d4-e325d16a869c" (UID: "24c2b816-5bf9-40f9-98d4-e325d16a869c"). InnerVolumeSpecName "kube-api-access-kpp5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:49:26.858752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.854910    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:26.858752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.854951    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpp5v\" (UniqueName: \"kubernetes.io/projected/24c2b816-5bf9-40f9-98d4-e325d16a869c-kube-api-access-kpp5v\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:26.858752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.854967    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c2b816-5bf9-40f9-98d4-e325d16a869c-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:26.858752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:26.854980    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24c2b816-5bf9-40f9-98d4-e325d16a869c-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:27.055698 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:27.055663    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"]
Nov 03 16:49:27.061039 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:27.061012    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-2bb620-d986cb659-fsjzm"]
Nov 03 16:49:28.309502 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:28.309469    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24c2b816-5bf9-40f9-98d4-e325d16a869c" path="/var/lib/kubelet/pods/24c2b816-5bf9-40f9-98d4-e325d16a869c/volumes"
Nov 03 16:49:31.703190 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:31.703150    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"]
Nov 03 16:49:33.394104 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:49:33.394061    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:49:35.110769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:35.110719    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"]
Nov 03 16:49:35.161971 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:35.161940    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"]
Nov 03 16:49:35.663346 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:35.663315    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"]
Nov 03 16:49:36.703831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:36.703765    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66" podUID="0a89c7b2-b9d2-4105-be8a-07232645205d" containerName="payload-processing" containerID="cri-o://d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1" gracePeriod=25
Nov 03 16:49:36.974522 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:36.974497    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:49:37.050935 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.050905    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a89c7b2-b9d2-4105-be8a-07232645205d-kube-api-access-x8fx6\" (UniqueName: \"kubernetes.io/projected/0a89c7b2-b9d2-4105-be8a-07232645205d-kube-api-access-x8fx6\") pod \"0a89c7b2-b9d2-4105-be8a-07232645205d\" (UID: \"0a89c7b2-b9d2-4105-be8a-07232645205d\") "
Nov 03 16:49:37.051102 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.050964    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a89c7b2-b9d2-4105-be8a-07232645205d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a89c7b2-b9d2-4105-be8a-07232645205d-plugins-config-volume\") pod \"0a89c7b2-b9d2-4105-be8a-07232645205d\" (UID: \"0a89c7b2-b9d2-4105-be8a-07232645205d\") "
Nov 03 16:49:37.051366 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.051342    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a89c7b2-b9d2-4105-be8a-07232645205d-plugins-config-volume" pod "0a89c7b2-b9d2-4105-be8a-07232645205d" (UID: "0a89c7b2-b9d2-4105-be8a-07232645205d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:49:37.053223 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.053203    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a89c7b2-b9d2-4105-be8a-07232645205d-kube-api-access-x8fx6" pod "0a89c7b2-b9d2-4105-be8a-07232645205d" (UID: "0a89c7b2-b9d2-4105-be8a-07232645205d"). InnerVolumeSpecName "kube-api-access-x8fx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:49:37.183184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.168278    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8fx6\" (UniqueName: \"kubernetes.io/projected/0a89c7b2-b9d2-4105-be8a-07232645205d-kube-api-access-x8fx6\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:37.183184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.168329    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a89c7b2-b9d2-4105-be8a-07232645205d-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:37.708587 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.708553    5605 generic.go:356] "Generic (PLEG): container finished" podID="0a89c7b2-b9d2-4105-be8a-07232645205d" containerID="d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1" exitCode=0
Nov 03 16:49:37.709062 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.708600    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"
Nov 03 16:49:37.709062 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.708629    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66" event={"ID":"0a89c7b2-b9d2-4105-be8a-07232645205d","Type":"ContainerDied","Data":"d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1"}
Nov 03 16:49:37.709062 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.708699    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66" event={"ID":"0a89c7b2-b9d2-4105-be8a-07232645205d","Type":"ContainerDied","Data":"1216156af15f96f93b024670b48636253e7d13a880b8d263c2204c0dc6fbe268"}
Nov 03 16:49:37.709062 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.708759    5605 scope.go:122] "RemoveContainer" containerID="d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1"
Nov 03 16:49:37.718887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.718870    5605 scope.go:122] "RemoveContainer" containerID="d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1"
Nov 03 16:49:37.719155 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:49:37.719138    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1\": container with ID starting with d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1 not found: ID does not exist" containerID="d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1"
Nov 03 16:49:37.719229 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.719162    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1"} err="failed to get container status \"d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1\": rpc error: code = NotFound desc = could not find container \"d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1\": container with ID starting with d7253c7dd39b9dca5357218bbab004fa03077663d7fed45bf25f7171596ee8c1 not found: ID does not exist"
Nov 03 16:49:37.918131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.914999    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"]
Nov 03 16:49:37.919614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:37.919582    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-66f145-b99dc85-bzg66"]
Nov 03 16:49:38.309026 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:38.308992    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a89c7b2-b9d2-4105-be8a-07232645205d" path="/var/lib/kubelet/pods/0a89c7b2-b9d2-4105-be8a-07232645205d/volumes"
Nov 03 16:49:40.663661 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:40.663614    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b" podUID="75eac520-9ac9-4564-a522-56cd310240c2" containerName="payload-pre-processing" containerID="cri-o://d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e" gracePeriod=25
Nov 03 16:49:40.924441 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:40.924385    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:49:41.030546 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.030512    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75eac520-9ac9-4564-a522-56cd310240c2-kube-api-access-ltzcb\" (UniqueName: \"kubernetes.io/projected/75eac520-9ac9-4564-a522-56cd310240c2-kube-api-access-ltzcb\") pod \"75eac520-9ac9-4564-a522-56cd310240c2\" (UID: \"75eac520-9ac9-4564-a522-56cd310240c2\") "
Nov 03 16:49:41.030691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.030643    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75eac520-9ac9-4564-a522-56cd310240c2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75eac520-9ac9-4564-a522-56cd310240c2-plugins-config-volume\") pod \"75eac520-9ac9-4564-a522-56cd310240c2\" (UID: \"75eac520-9ac9-4564-a522-56cd310240c2\") "
Nov 03 16:49:41.031026 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.031006    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75eac520-9ac9-4564-a522-56cd310240c2-plugins-config-volume" pod "75eac520-9ac9-4564-a522-56cd310240c2" (UID: "75eac520-9ac9-4564-a522-56cd310240c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:49:41.031289 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.031273    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75eac520-9ac9-4564-a522-56cd310240c2-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:41.032953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.032931    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75eac520-9ac9-4564-a522-56cd310240c2-kube-api-access-ltzcb" pod "75eac520-9ac9-4564-a522-56cd310240c2" (UID: "75eac520-9ac9-4564-a522-56cd310240c2"). InnerVolumeSpecName "kube-api-access-ltzcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:49:41.133068 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.133035    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltzcb\" (UniqueName: \"kubernetes.io/projected/75eac520-9ac9-4564-a522-56cd310240c2-kube-api-access-ltzcb\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:41.743382 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.743340    5605 generic.go:356] "Generic (PLEG): container finished" podID="75eac520-9ac9-4564-a522-56cd310240c2" containerID="d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e" exitCode=0
Nov 03 16:49:41.743795 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.743398    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"
Nov 03 16:49:41.743795 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.743397    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b" event={"ID":"75eac520-9ac9-4564-a522-56cd310240c2","Type":"ContainerDied","Data":"d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e"}
Nov 03 16:49:41.743795 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.743553    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b" event={"ID":"75eac520-9ac9-4564-a522-56cd310240c2","Type":"ContainerDied","Data":"0afab97c805d6a0a112374941dae383e71b54bda69b8dc6ce268c4e36d377e12"}
Nov 03 16:49:41.743795 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.743580    5605 scope.go:122] "RemoveContainer" containerID="d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e"
Nov 03 16:49:41.754008 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.753987    5605 scope.go:122] "RemoveContainer" containerID="d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e"
Nov 03 16:49:41.754295 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:49:41.754278    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e\": container with ID starting with d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e not found: ID does not exist" containerID="d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e"
Nov 03 16:49:41.754374 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.754301    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e"} err="failed to get container status \"d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e\": rpc error: code = NotFound desc = could not find container \"d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e\": container with ID starting with d6a991564e2f89b3be7beb65ee0f4e6e58db7d435bb10508c06ae36235ec340e not found: ID does not exist"
Nov 03 16:49:41.918931 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.918901    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"]
Nov 03 16:49:41.970339 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:41.970305    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c626cd-78bd99494c-b5x2b"]
Nov 03 16:49:42.309221 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:42.309184    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75eac520-9ac9-4564-a522-56cd310240c2" path="/var/lib/kubelet/pods/75eac520-9ac9-4564-a522-56cd310240c2/volumes"
Nov 03 16:49:50.131842 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.131776    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" containerID="cri-o://d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9" gracePeriod=45
Nov 03 16:49:50.183030 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.182983    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" containerID="cri-o://ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688" gracePeriod=45
Nov 03 16:49:50.338494 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.338158    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main" probeResult="failure" output="Get \"https://10.133.0.50:8000/ready\": dial tcp 10.133.0.50:8000: connect: connection refused"
Nov 03 16:49:50.345263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.345160    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": dial tcp 10.133.0.43:8000: connect: connection refused"
Nov 03 16:49:50.450592 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.450565    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:49:50.476419 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.476388    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-dshm\") pod \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") "
Nov 03 16:49:50.476555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.476471    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec4f033d-ff46-499b-bf4c-4712f04aaf31-kube-api-access-hxvkf\" (UniqueName: \"kubernetes.io/projected/ec4f033d-ff46-499b-bf4c-4712f04aaf31-kube-api-access-hxvkf\") pod \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") "
Nov 03 16:49:50.476555 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.476537    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tmp-dir\") pod \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") "
Nov 03 16:49:50.476694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.476586    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tls-certs\") pod \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") "
Nov 03 16:49:50.476694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.476672    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-model-cache\") pod \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") "
Nov 03 16:49:50.476833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.476779    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-home\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-home\") pod \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\" (UID: \"ec4f033d-ff46-499b-bf4c-4712f04aaf31\") "
Nov 03 16:49:50.476886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.476827    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tmp-dir" pod "ec4f033d-ff46-499b-bf4c-4712f04aaf31" (UID: "ec4f033d-ff46-499b-bf4c-4712f04aaf31"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:50.477028 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.476978    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-model-cache" pod "ec4f033d-ff46-499b-bf4c-4712f04aaf31" (UID: "ec4f033d-ff46-499b-bf4c-4712f04aaf31"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:50.477211 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.477052    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-home" pod "ec4f033d-ff46-499b-bf4c-4712f04aaf31" (UID: "ec4f033d-ff46-499b-bf4c-4712f04aaf31"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:50.477356 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.477340    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:49:50.477704 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.477681    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.477784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.477713    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.477784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.477749    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.479007 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.478986    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec4f033d-ff46-499b-bf4c-4712f04aaf31-kube-api-access-hxvkf" pod "ec4f033d-ff46-499b-bf4c-4712f04aaf31" (UID: "ec4f033d-ff46-499b-bf4c-4712f04aaf31"). InnerVolumeSpecName "kube-api-access-hxvkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:49:50.479265 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.479231    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-dshm" pod "ec4f033d-ff46-499b-bf4c-4712f04aaf31" (UID: "ec4f033d-ff46-499b-bf4c-4712f04aaf31"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:50.479265 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.479240    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tls-certs" pod "ec4f033d-ff46-499b-bf4c-4712f04aaf31" (UID: "ec4f033d-ff46-499b-bf4c-4712f04aaf31"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:49:50.578321 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.578281    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-tmp-dir\") pod \"7d0545c6-3e85-47d4-bf3e-b229191e424a\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") "
Nov 03 16:49:50.578519 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.578360    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-model-cache\") pod \"7d0545c6-3e85-47d4-bf3e-b229191e424a\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") "
Nov 03 16:49:50.578519 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.578486    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-home\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-home\") pod \"7d0545c6-3e85-47d4-bf3e-b229191e424a\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") "
Nov 03 16:49:50.578646 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.578547    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-dshm\") pod \"7d0545c6-3e85-47d4-bf3e-b229191e424a\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") "
Nov 03 16:49:50.578646 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.578552    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-tmp-dir" pod "7d0545c6-3e85-47d4-bf3e-b229191e424a" (UID: "7d0545c6-3e85-47d4-bf3e-b229191e424a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:50.578646 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.578592    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d0545c6-3e85-47d4-bf3e-b229191e424a-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d0545c6-3e85-47d4-bf3e-b229191e424a-tls-certs\") pod \"7d0545c6-3e85-47d4-bf3e-b229191e424a\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") "
Nov 03 16:49:50.578833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.578767    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d0545c6-3e85-47d4-bf3e-b229191e424a-kube-api-access-7kkwv\" (UniqueName: \"kubernetes.io/projected/7d0545c6-3e85-47d4-bf3e-b229191e424a-kube-api-access-7kkwv\") pod \"7d0545c6-3e85-47d4-bf3e-b229191e424a\" (UID: \"7d0545c6-3e85-47d4-bf3e-b229191e424a\") "
Nov 03 16:49:50.578833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.578769    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-home" pod "7d0545c6-3e85-47d4-bf3e-b229191e424a" (UID: "7d0545c6-3e85-47d4-bf3e-b229191e424a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:50.579302 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.579275    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec4f033d-ff46-499b-bf4c-4712f04aaf31-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.579431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.579303    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-model-cache" pod "7d0545c6-3e85-47d4-bf3e-b229191e424a" (UID: "7d0545c6-3e85-47d4-bf3e-b229191e424a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:50.579431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.579309    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxvkf\" (UniqueName: \"kubernetes.io/projected/ec4f033d-ff46-499b-bf4c-4712f04aaf31-kube-api-access-hxvkf\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.579431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.579387    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.579431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.579406    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4f033d-ff46-499b-bf4c-4712f04aaf31-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.579431 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.579418    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.582307 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.582248    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d0545c6-3e85-47d4-bf3e-b229191e424a-tls-certs" pod "7d0545c6-3e85-47d4-bf3e-b229191e424a" (UID: "7d0545c6-3e85-47d4-bf3e-b229191e424a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:49:50.582307 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.582271    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-dshm" pod "7d0545c6-3e85-47d4-bf3e-b229191e424a" (UID: "7d0545c6-3e85-47d4-bf3e-b229191e424a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:49:50.585848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.585812    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d0545c6-3e85-47d4-bf3e-b229191e424a-kube-api-access-7kkwv" pod "7d0545c6-3e85-47d4-bf3e-b229191e424a" (UID: "7d0545c6-3e85-47d4-bf3e-b229191e424a"). InnerVolumeSpecName "kube-api-access-7kkwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:49:50.680674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.680592    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.680674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.680618    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d0545c6-3e85-47d4-bf3e-b229191e424a-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.680674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.680630    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kkwv\" (UniqueName: \"kubernetes.io/projected/7d0545c6-3e85-47d4-bf3e-b229191e424a-kube-api-access-7kkwv\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.680674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.680643    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d0545c6-3e85-47d4-bf3e-b229191e424a-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:49:50.841860 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.841824    5605 generic.go:356] "Generic (PLEG): container finished" podID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerID="d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9" exitCode=0
Nov 03 16:49:50.842029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.841859    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" event={"ID":"ec4f033d-ff46-499b-bf4c-4712f04aaf31","Type":"ContainerDied","Data":"d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9"}
Nov 03 16:49:50.842029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.841910    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz" event={"ID":"ec4f033d-ff46-499b-bf4c-4712f04aaf31","Type":"ContainerDied","Data":"fcd3786f03c0b0a2bcdf9dec6f3f362dceba1390e7c42bf4a0e500e6d9f3215f"}
Nov 03 16:49:50.842029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.841916    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"
Nov 03 16:49:50.842029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.841931    5605 scope.go:122] "RemoveContainer" containerID="d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9"
Nov 03 16:49:50.843462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.843437    5605 generic.go:356] "Generic (PLEG): container finished" podID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerID="ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688" exitCode=0
Nov 03 16:49:50.843561 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.843521    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"
Nov 03 16:49:50.843630 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.843511    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" event={"ID":"7d0545c6-3e85-47d4-bf3e-b229191e424a","Type":"ContainerDied","Data":"ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688"}
Nov 03 16:49:50.843687 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.843630    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v" event={"ID":"7d0545c6-3e85-47d4-bf3e-b229191e424a","Type":"ContainerDied","Data":"60e7bfa83e9773c4b7796aa7de33d855a417d01092482b1c13704b73fc4961e3"}
Nov 03 16:49:50.852620 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.852600    5605 scope.go:122] "RemoveContainer" containerID="d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9"
Nov 03 16:49:50.852969 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:49:50.852949    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9\": container with ID starting with d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9 not found: ID does not exist" containerID="d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9"
Nov 03 16:49:50.853052 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.852976    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9"} err="failed to get container status \"d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9\": rpc error: code = NotFound desc = could not find container \"d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9\": container with ID starting with d85fae45b2d23ff2c7d74f07e90c1a8f89e654455538b001051314f98e9192c9 not found: ID does not exist"
Nov 03 16:49:50.853052 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.852995    5605 scope.go:122] "RemoveContainer" containerID="ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688"
Nov 03 16:49:50.862207 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.862186    5605 scope.go:122] "RemoveContainer" containerID="ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688"
Nov 03 16:49:50.862459 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:49:50.862441    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688\": container with ID starting with ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688 not found: ID does not exist" containerID="ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688"
Nov 03 16:49:50.862510 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:50.862467    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688"} err="failed to get container status \"ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688\": rpc error: code = NotFound desc = could not find container \"ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688\": container with ID starting with ea0173b80cfc4ac8be022a764841e85aec7cc4f033aaf33ef453142bc617f688 not found: ID does not exist"
Nov 03 16:49:51.026161 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:51.026074    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"]
Nov 03 16:49:51.032548 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:51.032519    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-2-w6-547a38-kserve-8544f969d-7jbgz"]
Nov 03 16:49:51.165536 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:51.162800    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"]
Nov 03 16:49:51.168837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:51.168628    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-547a38/distinct-w6-547a38-kserve-66765cf47c-p5v6v"]
Nov 03 16:49:52.309100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:52.309061    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" path="/var/lib/kubelet/pods/7d0545c6-3e85-47d4-bf3e-b229191e424a/volumes"
Nov 03 16:49:52.309648 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:52.309632    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" path="/var/lib/kubelet/pods/ec4f033d-ff46-499b-bf4c-4712f04aaf31/volumes"
Nov 03 16:49:59.958859 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:49:59.958771    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"]
Nov 03 16:50:01.509524 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.509484    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"]
Nov 03 16:50:01.510953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.510924    5605 cpu_manager.go:414] "containerMap: removing container" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy"
Nov 03 16:50:01.511132 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.511114    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy"
Nov 03 16:50:01.511250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.511236    5605 cpu_manager.go:414] "containerMap: removing container" podUID="d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb" containerName="maas-api"
Nov 03 16:50:01.511355 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.511341    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb" containerName="maas-api"
Nov 03 16:50:01.511466 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.511453    5605 cpu_manager.go:414] "containerMap: removing container" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy"
Nov 03 16:50:01.511558 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.511543    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy"
Nov 03 16:50:01.511655 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.511644    5605 cpu_manager.go:414] "containerMap: removing container" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main"
Nov 03 16:50:01.511762 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.511747    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main"
Nov 03 16:50:01.516754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.511852    5605 cpu_manager.go:414] "containerMap: removing container" podUID="0a89c7b2-b9d2-4105-be8a-07232645205d" containerName="payload-processing"
Nov 03 16:50:01.516950 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.516929    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a89c7b2-b9d2-4105-be8a-07232645205d" containerName="payload-processing"
Nov 03 16:50:01.517061 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.517048    5605 cpu_manager.go:414] "containerMap: removing container" podUID="24c2b816-5bf9-40f9-98d4-e325d16a869c" containerName="maas-api"
Nov 03 16:50:01.517149 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.517137    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24c2b816-5bf9-40f9-98d4-e325d16a869c" containerName="maas-api"
Nov 03 16:50:01.517249 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.517238    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main"
Nov 03 16:50:01.517334 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.517323    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main"
Nov 03 16:50:01.517446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.517434    5605 cpu_manager.go:414] "containerMap: removing container" podUID="75eac520-9ac9-4564-a522-56cd310240c2" containerName="payload-pre-processing"
Nov 03 16:50:01.517539 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.517526    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75eac520-9ac9-4564-a522-56cd310240c2" containerName="payload-pre-processing"
Nov 03 16:50:01.517833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.517813    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d0545c6-3e85-47d4-bf3e-b229191e424a" containerName="main"
Nov 03 16:50:01.530853 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.530826    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a89c7b2-b9d2-4105-be8a-07232645205d" containerName="payload-processing"
Nov 03 16:50:01.531042 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.531027    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="24c2b816-5bf9-40f9-98d4-e325d16a869c" containerName="maas-api"
Nov 03 16:50:01.531166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.531153    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec4f033d-ff46-499b-bf4c-4712f04aaf31" containerName="main"
Nov 03 16:50:01.531264 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.531252    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="75eac520-9ac9-4564-a522-56cd310240c2" containerName="payload-pre-processing"
Nov 03 16:50:01.531370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.531358    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy"
Nov 03 16:50:01.531461 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.531450    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="d32e79ed-90dd-402c-88a3-5e0ed5ffe2eb" containerName="maas-api"
Nov 03 16:50:01.531549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.531537    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="afb0f028-ecde-48ed-92b0-39276f7306e7" containerName="istio-proxy"
Nov 03 16:50:01.542749 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.537576    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.570385 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.570350    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"]
Nov 03 16:50:01.662750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.662554    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.662750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.662715    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-metrics-certs\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.662997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.662775    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzstk\" (UniqueName: \"kubernetes.io/projected/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-kube-api-access-wzstk\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.662997 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.662873    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-maas-api-tls\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.683752 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.678113    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-3b4d9b-serving-cert\""
Nov 03 16:50:01.793002 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.792904    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.793347 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.793321    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-metrics-certs\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.793490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.793473    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzstk\" (UniqueName: \"kubernetes.io/projected/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-kube-api-access-wzstk\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.793673 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.793657    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-maas-api-tls\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.794026 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:50:01.794000    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-serving-cert: secret "maas-api-e2e-inf-a-3b4d9b-serving-cert" not found
Nov 03 16:50:01.794233 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:50:01.794218    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-maas-api-tls podName:f9cacf9e-deaf-41a7-87ad-7915154f4ac8 nodeName:}" failed. No retries permitted until 2026-11-03 16:50:02.294192838 +0000 UTC m=+2288.118644103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-maas-api-tls") pod "maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx" (UID: "f9cacf9e-deaf-41a7-87ad-7915154f4ac8") : secret "maas-api-e2e-inf-a-3b4d9b-serving-cert" not found
Nov 03 16:50:01.795405 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.795376    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.930750 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.930318    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzstk\" (UniqueName: \"kubernetes.io/projected/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-kube-api-access-wzstk\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:01.952002 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:01.951932    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-metrics-certs\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:02.323792 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:02.323755    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-maas-api-tls\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:02.340817 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:02.339837    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-maas-api-tls\") pod \"maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:02.473835 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:02.472989    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:03.120753 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:03.111165    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"]
Nov 03 16:50:03.139920 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:50:03.139886    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9cacf9e_deaf_41a7_87ad_7915154f4ac8.slice/crio-ad803734ccf9ca0f09848f150371f181563bf1745be9e01d161706bd961696ed WatchSource:0}: Error finding container ad803734ccf9ca0f09848f150371f181563bf1745be9e01d161706bd961696ed: Status 404 returned error can't find the container with id ad803734ccf9ca0f09848f150371f181563bf1745be9e01d161706bd961696ed
Nov 03 16:50:03.421362 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:50:03.421254    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:50:03.978250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:03.978212    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx" event={"ID":"f9cacf9e-deaf-41a7-87ad-7915154f4ac8","Type":"ContainerStarted","Data":"ad803734ccf9ca0f09848f150371f181563bf1745be9e01d161706bd961696ed"}
Nov 03 16:50:05.013787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:05.013714    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx" event={"ID":"f9cacf9e-deaf-41a7-87ad-7915154f4ac8","Type":"ContainerStarted","Data":"c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413"}
Nov 03 16:50:05.014184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:05.013848    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:05.136744 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:05.136685    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx" podStartSLOduration=3.254875285 podStartE2EDuration="4.136671717s" totalImagesPullingTime="881.796432ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:50:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:50:05.135600601 +0000 UTC m=+2290.960051872" watchObservedRunningTime="2026-11-03 16:50:05.136671717 +0000 UTC m=+2290.961122987"
Nov 03 16:50:06.509324 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:06.509280    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"]
Nov 03 16:50:06.560470 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:06.560437    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"]
Nov 03 16:50:11.026287 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:11.026256    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:50:11.748509 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:11.748430    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"]
Nov 03 16:50:11.753678 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:11.753612    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q" podUID="62c29f8b-bd93-4ab0-ac1d-682247836f51" containerName="maas-api" containerID="cri-o://9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe" gracePeriod=30
Nov 03 16:50:12.051460 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.051428    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:50:12.056156 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.056126    5605 generic.go:356] "Generic (PLEG): container finished" podID="62c29f8b-bd93-4ab0-ac1d-682247836f51" containerID="9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe" exitCode=0
Nov 03 16:50:12.056306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.056173    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"
Nov 03 16:50:12.056306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.056207    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q" event={"ID":"62c29f8b-bd93-4ab0-ac1d-682247836f51","Type":"ContainerDied","Data":"9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe"}
Nov 03 16:50:12.056306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.056276    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q" event={"ID":"62c29f8b-bd93-4ab0-ac1d-682247836f51","Type":"ContainerDied","Data":"0d830bc4c07f383b5493bd14f7b26e151441cef4d7c00bb9b0ebc73853934c37"}
Nov 03 16:50:12.056306 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.056303    5605 scope.go:122] "RemoveContainer" containerID="9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe"
Nov 03 16:50:12.065682 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.065661    5605 scope.go:122] "RemoveContainer" containerID="9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe"
Nov 03 16:50:12.065998 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:50:12.065978    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe\": container with ID starting with 9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe not found: ID does not exist" containerID="9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe"
Nov 03 16:50:12.066056 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.066007    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe"} err="failed to get container status \"9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe\": rpc error: code = NotFound desc = could not find container \"9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe\": container with ID starting with 9b2c5150a35d016f065629ad586a0896577606643aa3cd21c967f9f2b97f1efe not found: ID does not exist"
Nov 03 16:50:12.191320 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.189795    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62c29f8b-bd93-4ab0-ac1d-682247836f51-kube-api-access-zpg47\" (UniqueName: \"kubernetes.io/projected/62c29f8b-bd93-4ab0-ac1d-682247836f51-kube-api-access-zpg47\") pod \"62c29f8b-bd93-4ab0-ac1d-682247836f51\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") "
Nov 03 16:50:12.191320 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.189865    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-maas-api-tls\") pod \"62c29f8b-bd93-4ab0-ac1d-682247836f51\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") "
Nov 03 16:50:12.191320 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.189916    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-metrics-certs\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-metrics-certs\") pod \"62c29f8b-bd93-4ab0-ac1d-682247836f51\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") "
Nov 03 16:50:12.191320 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.190132    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62c29f8b-bd93-4ab0-ac1d-682247836f51-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c29f8b-bd93-4ab0-ac1d-682247836f51-openshift-service-ca\") pod \"62c29f8b-bd93-4ab0-ac1d-682247836f51\" (UID: \"62c29f8b-bd93-4ab0-ac1d-682247836f51\") "
Nov 03 16:50:12.191320 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.190935    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62c29f8b-bd93-4ab0-ac1d-682247836f51-openshift-service-ca" pod "62c29f8b-bd93-4ab0-ac1d-682247836f51" (UID: "62c29f8b-bd93-4ab0-ac1d-682247836f51"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:50:12.198839 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.198812    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-maas-api-tls" pod "62c29f8b-bd93-4ab0-ac1d-682247836f51" (UID: "62c29f8b-bd93-4ab0-ac1d-682247836f51"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:50:12.199026 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.198908    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-metrics-certs" pod "62c29f8b-bd93-4ab0-ac1d-682247836f51" (UID: "62c29f8b-bd93-4ab0-ac1d-682247836f51"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:50:12.199026 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.198951    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62c29f8b-bd93-4ab0-ac1d-682247836f51-kube-api-access-zpg47" pod "62c29f8b-bd93-4ab0-ac1d-682247836f51" (UID: "62c29f8b-bd93-4ab0-ac1d-682247836f51"). InnerVolumeSpecName "kube-api-access-zpg47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:50:12.291757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.291654    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpg47\" (UniqueName: \"kubernetes.io/projected/62c29f8b-bd93-4ab0-ac1d-682247836f51-kube-api-access-zpg47\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:12.291757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.291684    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:12.291757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.291698    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62c29f8b-bd93-4ab0-ac1d-682247836f51-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:12.291757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.291711    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62c29f8b-bd93-4ab0-ac1d-682247836f51-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:12.477308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.477269    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"]
Nov 03 16:50:12.482085 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:12.482056    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c626cd-58cf775875-qwq8q"]
Nov 03 16:50:14.327783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:14.327682    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62c29f8b-bd93-4ab0-ac1d-682247836f51" path="/var/lib/kubelet/pods/62c29f8b-bd93-4ab0-ac1d-682247836f51/volumes"
Nov 03 16:50:14.980928 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:14.980890    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main" containerID="cri-o://5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6" gracePeriod=45
Nov 03 16:50:15.251638 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.251614    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:50:15.334986 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.334954    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-model-cache\") pod \"7e014c82-88ba-42be-9a82-ae0367902465\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") "
Nov 03 16:50:15.335414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335052    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-home\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-home\") pod \"7e014c82-88ba-42be-9a82-ae0367902465\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") "
Nov 03 16:50:15.335414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335123    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-tmp-dir\") pod \"7e014c82-88ba-42be-9a82-ae0367902465\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") "
Nov 03 16:50:15.335414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335205    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-dshm\") pod \"7e014c82-88ba-42be-9a82-ae0367902465\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") "
Nov 03 16:50:15.335414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335294    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-model-cache" pod "7e014c82-88ba-42be-9a82-ae0367902465" (UID: "7e014c82-88ba-42be-9a82-ae0367902465"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:15.335414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335301    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7e014c82-88ba-42be-9a82-ae0367902465-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e014c82-88ba-42be-9a82-ae0367902465-tls-certs\") pod \"7e014c82-88ba-42be-9a82-ae0367902465\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") "
Nov 03 16:50:15.335414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335316    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-tmp-dir" pod "7e014c82-88ba-42be-9a82-ae0367902465" (UID: "7e014c82-88ba-42be-9a82-ae0367902465"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:15.335414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335348    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-home" pod "7e014c82-88ba-42be-9a82-ae0367902465" (UID: "7e014c82-88ba-42be-9a82-ae0367902465"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:15.335414 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335373    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e014c82-88ba-42be-9a82-ae0367902465-kube-api-access-25xpq\" (UniqueName: \"kubernetes.io/projected/7e014c82-88ba-42be-9a82-ae0367902465-kube-api-access-25xpq\") pod \"7e014c82-88ba-42be-9a82-ae0367902465\" (UID: \"7e014c82-88ba-42be-9a82-ae0367902465\") "
Nov 03 16:50:15.335972 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335900    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:15.335972 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335927    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:15.335972 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.335937    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:15.337600 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.337568    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e014c82-88ba-42be-9a82-ae0367902465-tls-certs" pod "7e014c82-88ba-42be-9a82-ae0367902465" (UID: "7e014c82-88ba-42be-9a82-ae0367902465"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:50:15.337600 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.337572    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-dshm" pod "7e014c82-88ba-42be-9a82-ae0367902465" (UID: "7e014c82-88ba-42be-9a82-ae0367902465"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:15.337778 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.337699    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e014c82-88ba-42be-9a82-ae0367902465-kube-api-access-25xpq" pod "7e014c82-88ba-42be-9a82-ae0367902465" (UID: "7e014c82-88ba-42be-9a82-ae0367902465"). InnerVolumeSpecName "kube-api-access-25xpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:50:15.437475 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.437439    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e014c82-88ba-42be-9a82-ae0367902465-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:15.437475 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.437469    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e014c82-88ba-42be-9a82-ae0367902465-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:15.437475 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:15.437480    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25xpq\" (UniqueName: \"kubernetes.io/projected/7e014c82-88ba-42be-9a82-ae0367902465-kube-api-access-25xpq\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:16.082930 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:16.082897    5605 generic.go:356] "Generic (PLEG): container finished" podID="7e014c82-88ba-42be-9a82-ae0367902465" containerID="5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6" exitCode=0
Nov 03 16:50:16.083123 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:16.082943    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"
Nov 03 16:50:16.083123 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:16.082988    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" event={"ID":"7e014c82-88ba-42be-9a82-ae0367902465","Type":"ContainerDied","Data":"5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6"}
Nov 03 16:50:16.083123 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:16.083023    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj" event={"ID":"7e014c82-88ba-42be-9a82-ae0367902465","Type":"ContainerDied","Data":"cdb496db3951a1ad806aab3cd5ba0f8bb9262fdf89a7d7cb4f02313f1a383cb2"}
Nov 03 16:50:16.083123 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:16.083041    5605 scope.go:122] "RemoveContainer" containerID="5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6"
Nov 03 16:50:16.096038 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:16.096018    5605 scope.go:122] "RemoveContainer" containerID="5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6"
Nov 03 16:50:16.096305 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:50:16.096282    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6\": container with ID starting with 5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6 not found: ID does not exist" containerID="5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6"
Nov 03 16:50:16.096379 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:16.096312    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6"} err="failed to get container status \"5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6\": rpc error: code = NotFound desc = could not find container \"5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6\": container with ID starting with 5314f989715db4b7cc3a6fcbd8ba161244aef230808cbe1158b003a9768ab0d6 not found: ID does not exist"
Nov 03 16:50:16.256498 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:16.256463    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"]
Nov 03 16:50:16.311021 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:16.310990    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-192564/premium-w3-192564-kserve-58cff55cbd-x75wj"]
Nov 03 16:50:18.310930 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:18.310886    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e014c82-88ba-42be-9a82-ae0367902465" path="/var/lib/kubelet/pods/7e014c82-88ba-42be-9a82-ae0367902465/volumes"
Nov 03 16:50:21.530533 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.530469    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main" containerID="cri-o://c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49" gracePeriod=45
Nov 03 16:50:21.580889 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.580850    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main" containerID="cri-o://04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b" gracePeriod=45
Nov 03 16:50:21.837132 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.837104    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:50:21.865418 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.865395    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:50:21.936117 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936090    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e552fb30-712d-4a82-9ddc-7af234dc9122-tls-certs\" (UniqueName: \"kubernetes.io/secret/e552fb30-712d-4a82-9ddc-7af234dc9122-tls-certs\") pod \"e552fb30-712d-4a82-9ddc-7af234dc9122\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") "
Nov 03 16:50:21.936308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936150    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-model-cache\") pod \"33b210d8-b8be-4b1a-b366-166dcec474ca\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") "
Nov 03 16:50:21.936308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936176    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-tmp-dir\") pod \"33b210d8-b8be-4b1a-b366-166dcec474ca\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") "
Nov 03 16:50:21.936308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936278    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e552fb30-712d-4a82-9ddc-7af234dc9122-kube-api-access-5r7s7\" (UniqueName: \"kubernetes.io/projected/e552fb30-712d-4a82-9ddc-7af234dc9122-kube-api-access-5r7s7\") pod \"e552fb30-712d-4a82-9ddc-7af234dc9122\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") "
Nov 03 16:50:21.936490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936330    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-dshm\") pod \"e552fb30-712d-4a82-9ddc-7af234dc9122\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") "
Nov 03 16:50:21.936490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936362    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/33b210d8-b8be-4b1a-b366-166dcec474ca-tls-certs\" (UniqueName: \"kubernetes.io/secret/33b210d8-b8be-4b1a-b366-166dcec474ca-tls-certs\") pod \"33b210d8-b8be-4b1a-b366-166dcec474ca\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") "
Nov 03 16:50:21.936490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936384    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-tmp-dir\") pod \"e552fb30-712d-4a82-9ddc-7af234dc9122\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") "
Nov 03 16:50:21.936490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936401    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-tmp-dir" pod "33b210d8-b8be-4b1a-b366-166dcec474ca" (UID: "33b210d8-b8be-4b1a-b366-166dcec474ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:21.936490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936407    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-model-cache" pod "33b210d8-b8be-4b1a-b366-166dcec474ca" (UID: "33b210d8-b8be-4b1a-b366-166dcec474ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:21.936490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936449    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-dshm\") pod \"33b210d8-b8be-4b1a-b366-166dcec474ca\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") "
Nov 03 16:50:21.936831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936522    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33b210d8-b8be-4b1a-b366-166dcec474ca-kube-api-access-92pcz\" (UniqueName: \"kubernetes.io/projected/33b210d8-b8be-4b1a-b366-166dcec474ca-kube-api-access-92pcz\") pod \"33b210d8-b8be-4b1a-b366-166dcec474ca\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") "
Nov 03 16:50:21.936831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936605    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-home\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-home\") pod \"33b210d8-b8be-4b1a-b366-166dcec474ca\" (UID: \"33b210d8-b8be-4b1a-b366-166dcec474ca\") "
Nov 03 16:50:21.936831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936634    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-model-cache\") pod \"e552fb30-712d-4a82-9ddc-7af234dc9122\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") "
Nov 03 16:50:21.936831 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936707    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-home\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-home\") pod \"e552fb30-712d-4a82-9ddc-7af234dc9122\" (UID: \"e552fb30-712d-4a82-9ddc-7af234dc9122\") "
Nov 03 16:50:21.937050 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.936826    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-tmp-dir" pod "e552fb30-712d-4a82-9ddc-7af234dc9122" (UID: "e552fb30-712d-4a82-9ddc-7af234dc9122"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:21.937259 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.937200    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-home" pod "33b210d8-b8be-4b1a-b366-166dcec474ca" (UID: "33b210d8-b8be-4b1a-b366-166dcec474ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:21.937577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.937414    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-home" pod "e552fb30-712d-4a82-9ddc-7af234dc9122" (UID: "e552fb30-712d-4a82-9ddc-7af234dc9122"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:21.937577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.937533    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:21.937577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.937556    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:21.937577 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.937570    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:21.937956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.937614    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:21.938100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.938070    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-model-cache" pod "e552fb30-712d-4a82-9ddc-7af234dc9122" (UID: "e552fb30-712d-4a82-9ddc-7af234dc9122"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:21.938936 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.938910    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33b210d8-b8be-4b1a-b366-166dcec474ca-tls-certs" pod "33b210d8-b8be-4b1a-b366-166dcec474ca" (UID: "33b210d8-b8be-4b1a-b366-166dcec474ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:50:21.938936 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.938916    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-dshm" pod "e552fb30-712d-4a82-9ddc-7af234dc9122" (UID: "e552fb30-712d-4a82-9ddc-7af234dc9122"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:21.939121 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.939094    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e552fb30-712d-4a82-9ddc-7af234dc9122-tls-certs" pod "e552fb30-712d-4a82-9ddc-7af234dc9122" (UID: "e552fb30-712d-4a82-9ddc-7af234dc9122"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:50:21.939186 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.939169    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-dshm" pod "33b210d8-b8be-4b1a-b366-166dcec474ca" (UID: "33b210d8-b8be-4b1a-b366-166dcec474ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:50:21.939413 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.939390    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e552fb30-712d-4a82-9ddc-7af234dc9122-kube-api-access-5r7s7" pod "e552fb30-712d-4a82-9ddc-7af234dc9122" (UID: "e552fb30-712d-4a82-9ddc-7af234dc9122"). InnerVolumeSpecName "kube-api-access-5r7s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:50:21.939512 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:21.939409    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33b210d8-b8be-4b1a-b366-166dcec474ca-kube-api-access-92pcz" pod "33b210d8-b8be-4b1a-b366-166dcec474ca" (UID: "33b210d8-b8be-4b1a-b366-166dcec474ca"). InnerVolumeSpecName "kube-api-access-92pcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:50:22.038995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.038961    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e552fb30-712d-4a82-9ddc-7af234dc9122-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:22.038995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.038989    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r7s7\" (UniqueName: \"kubernetes.io/projected/e552fb30-712d-4a82-9ddc-7af234dc9122-kube-api-access-5r7s7\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:22.038995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.038999    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:22.039236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.039007    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b210d8-b8be-4b1a-b366-166dcec474ca-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:22.039236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.039015    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b210d8-b8be-4b1a-b366-166dcec474ca-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:22.039236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.039023    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92pcz\" (UniqueName: \"kubernetes.io/projected/33b210d8-b8be-4b1a-b366-166dcec474ca-kube-api-access-92pcz\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:22.039236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.039031    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:22.039236 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.039040    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e552fb30-712d-4a82-9ddc-7af234dc9122-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:50:22.127501 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.127463    5605 generic.go:356] "Generic (PLEG): container finished" podID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerID="c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49" exitCode=0
Nov 03 16:50:22.127691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.127529    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"
Nov 03 16:50:22.127691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.127555    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" event={"ID":"e552fb30-712d-4a82-9ddc-7af234dc9122","Type":"ContainerDied","Data":"c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49"}
Nov 03 16:50:22.127691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.127622    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66" event={"ID":"e552fb30-712d-4a82-9ddc-7af234dc9122","Type":"ContainerDied","Data":"0f8662978c2998e78f92e66366615258d36fff74685290b20e006d5e731f6786"}
Nov 03 16:50:22.127691 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.127645    5605 scope.go:122] "RemoveContainer" containerID="c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49"
Nov 03 16:50:22.129804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.129651    5605 generic.go:356] "Generic (PLEG): container finished" podID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerID="04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b" exitCode=0
Nov 03 16:50:22.129804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.129704    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" event={"ID":"33b210d8-b8be-4b1a-b366-166dcec474ca","Type":"ContainerDied","Data":"04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b"}
Nov 03 16:50:22.129804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.129769    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc" event={"ID":"33b210d8-b8be-4b1a-b366-166dcec474ca","Type":"ContainerDied","Data":"d1a8f7edfe18f71448ff78bfdd7ba76eb7f489941137084c48d10798bf723f57"}
Nov 03 16:50:22.129804 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.129771    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"
Nov 03 16:50:22.140021 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.139951    5605 scope.go:122] "RemoveContainer" containerID="c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49"
Nov 03 16:50:22.140343 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:50:22.140310    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49\": container with ID starting with c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49 not found: ID does not exist" containerID="c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49"
Nov 03 16:50:22.140440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.140353    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49"} err="failed to get container status \"c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49\": rpc error: code = NotFound desc = could not find container \"c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49\": container with ID starting with c87a17e10cb48876d7a15d1004f7bf36bc1655d8c3b3581e2625b652be916f49 not found: ID does not exist"
Nov 03 16:50:22.140440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.140377    5605 scope.go:122] "RemoveContainer" containerID="04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b"
Nov 03 16:50:22.150379 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.150357    5605 scope.go:122] "RemoveContainer" containerID="04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b"
Nov 03 16:50:22.150677 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:50:22.150657    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b\": container with ID starting with 04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b not found: ID does not exist" containerID="04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b"
Nov 03 16:50:22.150844 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.150688    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b"} err="failed to get container status \"04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b\": rpc error: code = NotFound desc = could not find container \"04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b\": container with ID starting with 04e9ee49f356d18b399b932ddc18eefe9a5d0ab4b7c2a8a6ab7a46becee6432b not found: ID does not exist"
Nov 03 16:50:22.300245 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.300214    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"]
Nov 03 16:50:22.309769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.309743    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/distinct-w5-6ae690-kserve-bb4c74c99-mkt66"]
Nov 03 16:50:22.424951 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.424871    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"]
Nov 03 16:50:22.431772 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:22.431740    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-6ae690/premium-w5-6ae690-kserve-759944fcb-zhjvc"]
Nov 03 16:50:24.309833 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:24.309803    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" path="/var/lib/kubelet/pods/33b210d8-b8be-4b1a-b366-166dcec474ca/volumes"
Nov 03 16:50:24.310313 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:24.310248    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" path="/var/lib/kubelet/pods/e552fb30-712d-4a82-9ddc-7af234dc9122/volumes"
Nov 03 16:50:33.393745 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:50:33.393690    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:50:34.366106 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366074    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"]
Nov 03 16:50:34.366587 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366574    5605 cpu_manager.go:414] "containerMap: removing container" podUID="62c29f8b-bd93-4ab0-ac1d-682247836f51" containerName="maas-api"
Nov 03 16:50:34.366642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366588    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62c29f8b-bd93-4ab0-ac1d-682247836f51" containerName="maas-api"
Nov 03 16:50:34.366642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366611    5605 cpu_manager.go:414] "containerMap: removing container" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main"
Nov 03 16:50:34.366642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366616    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main"
Nov 03 16:50:34.366642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366625    5605 cpu_manager.go:414] "containerMap: removing container" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main"
Nov 03 16:50:34.366642 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366629    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main"
Nov 03 16:50:34.366846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366645    5605 cpu_manager.go:414] "containerMap: removing container" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main"
Nov 03 16:50:34.366846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366650    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main"
Nov 03 16:50:34.366846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366740    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e014c82-88ba-42be-9a82-ae0367902465" containerName="main"
Nov 03 16:50:34.366846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366754    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="62c29f8b-bd93-4ab0-ac1d-682247836f51" containerName="maas-api"
Nov 03 16:50:34.366846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366763    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="33b210d8-b8be-4b1a-b366-166dcec474ca" containerName="main"
Nov 03 16:50:34.366846 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.366770    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="e552fb30-712d-4a82-9ddc-7af234dc9122" containerName="main"
Nov 03 16:50:34.371384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.371367    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:50:34.410276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.410240    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t\" (UID: \"f7cb22e7-2865-4211-a1a4-12eba70bd7b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:50:34.410680 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.410466    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2lc8\" (UniqueName: \"kubernetes.io/projected/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-kube-api-access-v2lc8\") pod \"payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t\" (UID: \"f7cb22e7-2865-4211-a1a4-12eba70bd7b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:50:34.428050 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.428018    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"]
Nov 03 16:50:34.477793 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.477758    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-7e55c3\""
Nov 03 16:50:34.488392 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.488339    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-7e55c3-dockercfg-2r85q\""
Nov 03 16:50:34.511605 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.511563    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2lc8\" (UniqueName: \"kubernetes.io/projected/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-kube-api-access-v2lc8\") pod \"payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t\" (UID: \"f7cb22e7-2865-4211-a1a4-12eba70bd7b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:50:34.512296 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.512266    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t\" (UID: \"f7cb22e7-2865-4211-a1a4-12eba70bd7b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:50:34.529757 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.513450    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t\" (UID: \"f7cb22e7-2865-4211-a1a4-12eba70bd7b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:50:34.535797 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.534296    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"]
Nov 03 16:50:34.551953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.551923    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:50:34.581794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.579640    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"]
Nov 03 16:50:34.599308 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.599275    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2lc8\" (UniqueName: \"kubernetes.io/projected/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-kube-api-access-v2lc8\") pod \"payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t\" (UID: \"f7cb22e7-2865-4211-a1a4-12eba70bd7b2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:50:34.613395 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.613363    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8kjj\" (UniqueName: \"kubernetes.io/projected/8c1cccb3-fd69-413b-9974-b1d1d9c27899-kube-api-access-g8kjj\") pod \"payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk\" (UID: \"8c1cccb3-fd69-413b-9974-b1d1d9c27899\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:50:34.613581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.613436    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c1cccb3-fd69-413b-9974-b1d1d9c27899-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk\" (UID: \"8c1cccb3-fd69-413b-9974-b1d1d9c27899\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:50:34.696082 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.695988    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:50:34.756269 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.756229    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c1cccb3-fd69-413b-9974-b1d1d9c27899-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk\" (UID: \"8c1cccb3-fd69-413b-9974-b1d1d9c27899\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:50:34.756706 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.756679    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8kjj\" (UniqueName: \"kubernetes.io/projected/8c1cccb3-fd69-413b-9974-b1d1d9c27899-kube-api-access-g8kjj\") pod \"payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk\" (UID: \"8c1cccb3-fd69-413b-9974-b1d1d9c27899\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:50:34.757258 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.757128    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c1cccb3-fd69-413b-9974-b1d1d9c27899-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk\" (UID: \"8c1cccb3-fd69-413b-9974-b1d1d9c27899\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:50:34.828008 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.827946    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8kjj\" (UniqueName: \"kubernetes.io/projected/8c1cccb3-fd69-413b-9974-b1d1d9c27899-kube-api-access-g8kjj\") pod \"payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk\" (UID: \"8c1cccb3-fd69-413b-9974-b1d1d9c27899\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:50:34.882229 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:34.882196    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:50:35.132327 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:35.132301    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"]
Nov 03 16:50:35.134231 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:50:35.134202    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7cb22e7_2865_4211_a1a4_12eba70bd7b2.slice/crio-a7fb6a7ffa845b4c9a516ab421104c04a2740c494259a64d22026e48083facd8 WatchSource:0}: Error finding container a7fb6a7ffa845b4c9a516ab421104c04a2740c494259a64d22026e48083facd8: Status 404 returned error can't find the container with id a7fb6a7ffa845b4c9a516ab421104c04a2740c494259a64d22026e48083facd8
Nov 03 16:50:35.300066 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:50:35.289143    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c1cccb3_fd69_413b_9974_b1d1d9c27899.slice/crio-6e6bb2702c7c70e0855bcd9609fe081db5d73340a3d3c8ea2d3aa87e7230ff35 WatchSource:0}: Error finding container 6e6bb2702c7c70e0855bcd9609fe081db5d73340a3d3c8ea2d3aa87e7230ff35: Status 404 returned error can't find the container with id 6e6bb2702c7c70e0855bcd9609fe081db5d73340a3d3c8ea2d3aa87e7230ff35
Nov 03 16:50:35.315446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:35.314538    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t" event={"ID":"f7cb22e7-2865-4211-a1a4-12eba70bd7b2","Type":"ContainerStarted","Data":"64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204"}
Nov 03 16:50:35.315446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:35.314595    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t" event={"ID":"f7cb22e7-2865-4211-a1a4-12eba70bd7b2","Type":"ContainerStarted","Data":"a7fb6a7ffa845b4c9a516ab421104c04a2740c494259a64d22026e48083facd8"}
Nov 03 16:50:35.315446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:35.315391    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:50:35.323286 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:35.323244    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"]
Nov 03 16:50:35.449970 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:35.449878    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t" podStartSLOduration=1.449864383 podStartE2EDuration="1.449864383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:50:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:50:35.44901046 +0000 UTC m=+2321.273461731" watchObservedRunningTime="2026-11-03 16:50:35.449864383 +0000 UTC m=+2321.274315655"
Nov 03 16:50:36.344158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:36.344122    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk" event={"ID":"8c1cccb3-fd69-413b-9974-b1d1d9c27899","Type":"ContainerStarted","Data":"f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f"}
Nov 03 16:50:36.344384 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:36.344181    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk" event={"ID":"8c1cccb3-fd69-413b-9974-b1d1d9c27899","Type":"ContainerStarted","Data":"6e6bb2702c7c70e0855bcd9609fe081db5d73340a3d3c8ea2d3aa87e7230ff35"}
Nov 03 16:50:36.466487 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:36.466432    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk" podStartSLOduration=2.46641797 podStartE2EDuration="2.46641797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:50:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:50:36.464600147 +0000 UTC m=+2322.289051419" watchObservedRunningTime="2026-11-03 16:50:36.46641797 +0000 UTC m=+2322.290869234"
Nov 03 16:50:37.350591 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:37.350556    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:50:46.417751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:46.417391    5605 generic.go:356] "Generic (PLEG): container finished" podID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerID="af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83" exitCode=0
Nov 03 16:50:46.417751 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:46.417445    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" event={"ID":"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96","Type":"ContainerDied","Data":"af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83"}
Nov 03 16:50:46.418370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:46.417943    5605 scope.go:122] "RemoveContainer" containerID="af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83"
Nov 03 16:50:46.420444 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:46.420402    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:50:46.420579 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:46.420511    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:50:46.420579 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:46.420564    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:50:47.353683 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:47.353649    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:50:47.425202 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:47.425169    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" event={"ID":"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96","Type":"ContainerStarted","Data":"9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec"}
Nov 03 16:50:48.046285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:48.046252    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:50:48.046478 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:48.046429    5605 patch_prober.go:29] interesting pod/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 03 16:50:48.046532 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:48.046476    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 03 16:50:48.360386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:48.360351    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:50:49.046216 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:49.046177    5605 patch_prober.go:29] interesting pod/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 03 16:50:49.046622 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:49.046241    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 03 16:50:50.046252 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:50.046210    5605 patch_prober.go:29] interesting pod/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 03 16:50:50.046661 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:50.046277    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 03 16:50:51.045938 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:51.045899    5605 patch_prober.go:29] interesting pod/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 03 16:50:51.046117 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:51.045952    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 03 16:50:52.046425 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:52.046387    5605 patch_prober.go:29] interesting pod/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 03 16:50:52.046840 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:52.046451    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 03 16:50:53.046169 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:53.046134    5605 patch_prober.go:29] interesting pod/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Nov 03 16:50:53.046433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:53.046255    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Nov 03 16:50:54.050410 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:54.050379    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:50:54.050848 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:54.050665    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:50:54.051408 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:50:54.051389    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:51:03.393825 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:51:03.393791    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:51:19.689353 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:19.689319    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"]
Nov 03 16:51:19.689930 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:19.689535    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy" containerID="cri-o://5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4" gracePeriod=30
Nov 03 16:51:20.029783 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.029665    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"]
Nov 03 16:51:20.033655 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.033636    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.054809 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.054780    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-maas-api-tls\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.054956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.054831    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-metrics-certs\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.055029 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.054952    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4250677-5a92-4fe9-88d5-66670ff0dfff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.055096 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.055049    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfg6j\" (UniqueName: \"kubernetes.io/projected/a4250677-5a92-4fe9-88d5-66670ff0dfff-kube-api-access-cfg6j\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.091474 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.091441    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7e55c3-serving-cert\""
Nov 03 16:51:20.092784 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.092757    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"]
Nov 03 16:51:20.156386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.156347    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-maas-api-tls\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.156635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.156423    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-metrics-certs\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.156635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.156485    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4250677-5a92-4fe9-88d5-66670ff0dfff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.156635 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:51:20.156539    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-serving-cert: secret "maas-api-e2e-inf-a-7e55c3-serving-cert" not found
Nov 03 16:51:20.156635 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:51:20.156620    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-maas-api-tls podName:a4250677-5a92-4fe9-88d5-66670ff0dfff nodeName:}" failed. No retries permitted until 2026-11-03 16:51:20.656599537 +0000 UTC m=+2366.481050806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-maas-api-tls") pod "maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn" (UID: "a4250677-5a92-4fe9-88d5-66670ff0dfff") : secret "maas-api-e2e-inf-a-7e55c3-serving-cert" not found
Nov 03 16:51:20.156635 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.156542    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfg6j\" (UniqueName: \"kubernetes.io/projected/a4250677-5a92-4fe9-88d5-66670ff0dfff-kube-api-access-cfg6j\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.157631 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.157606    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4250677-5a92-4fe9-88d5-66670ff0dfff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.159818 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.159796    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-metrics-certs\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.220131 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.220104    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfg6j\" (UniqueName: \"kubernetes.io/projected/a4250677-5a92-4fe9-88d5-66670ff0dfff-kube-api-access-cfg6j\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.667209 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.667171    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-maas-api-tls\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.669754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.669699    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-maas-api-tls\") pod \"maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:20.947429 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:20.947328    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:21.331553 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:21.331528    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"]
Nov 03 16:51:21.333140 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:51:21.333112    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4250677_5a92_4fe9_88d5_66670ff0dfff.slice/crio-78429fa14ef3d73cdcf3cdd7de6ce7543c70d71a3d9c5c98be89393787b6ec14 WatchSource:0}: Error finding container 78429fa14ef3d73cdcf3cdd7de6ce7543c70d71a3d9c5c98be89393787b6ec14: Status 404 returned error can't find the container with id 78429fa14ef3d73cdcf3cdd7de6ce7543c70d71a3d9c5c98be89393787b6ec14
Nov 03 16:51:21.632470 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:21.632437    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn" event={"ID":"a4250677-5a92-4fe9-88d5-66670ff0dfff","Type":"ContainerStarted","Data":"78429fa14ef3d73cdcf3cdd7de6ce7543c70d71a3d9c5c98be89393787b6ec14"}
Nov 03 16:51:22.639710 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:22.639674    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn" event={"ID":"a4250677-5a92-4fe9-88d5-66670ff0dfff","Type":"ContainerStarted","Data":"e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2"}
Nov 03 16:51:22.640139 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:22.639763    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:22.762594 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:22.762513    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn" podStartSLOduration=2.898966214 podStartE2EDuration="3.762493949s" totalImagesPullingTime="863.527735ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:51:22.760661911 +0000 UTC m=+2368.585113205" watchObservedRunningTime="2026-11-03 16:51:22.762493949 +0000 UTC m=+2368.586945222"
Nov 03 16:51:24.304296 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:24.304255    5605 patch_prober.go:29] interesting pod/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:51:24.304766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:24.304323    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:51:24.964117 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:24.964052    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:51:25.036440 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036406    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-socket\") pod \"f5748322-abad-483c-ba35-10e1178e5a4d\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") "
Nov 03 16:51:25.036584 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036458    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-kube-api-access-xp26s\" (UniqueName: \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-kube-api-access-xp26s\") pod \"f5748322-abad-483c-ba35-10e1178e5a4d\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") "
Nov 03 16:51:25.036632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036585    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-credential-socket\") pod \"f5748322-abad-483c-ba35-10e1178e5a4d\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") "
Nov 03 16:51:25.036632 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036621    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-envoy\") pod \"f5748322-abad-483c-ba35-10e1178e5a4d\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") "
Nov 03 16:51:25.036781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036662    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-istio-token\" (UniqueName: \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-istio-token\") pod \"f5748322-abad-483c-ba35-10e1178e5a4d\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") "
Nov 03 16:51:25.036781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036707    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-data\") pod \"f5748322-abad-483c-ba35-10e1178e5a4d\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") "
Nov 03 16:51:25.036781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036768    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f5748322-abad-483c-ba35-10e1178e5a4d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5748322-abad-483c-ba35-10e1178e5a4d-istio-podinfo\") pod \"f5748322-abad-483c-ba35-10e1178e5a4d\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") "
Nov 03 16:51:25.036954 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036804    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-certs\") pod \"f5748322-abad-483c-ba35-10e1178e5a4d\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") "
Nov 03 16:51:25.036954 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036822    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5748322-abad-483c-ba35-10e1178e5a4d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5748322-abad-483c-ba35-10e1178e5a4d-istiod-ca-cert\") pod \"f5748322-abad-483c-ba35-10e1178e5a4d\" (UID: \"f5748322-abad-483c-ba35-10e1178e5a4d\") "
Nov 03 16:51:25.036954 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036715    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-socket" pod "f5748322-abad-483c-ba35-10e1178e5a4d" (UID: "f5748322-abad-483c-ba35-10e1178e5a4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:51:25.037098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036927    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-credential-socket" pod "f5748322-abad-483c-ba35-10e1178e5a4d" (UID: "f5748322-abad-483c-ba35-10e1178e5a4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:51:25.037098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.036999    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-data" pod "f5748322-abad-483c-ba35-10e1178e5a4d" (UID: "f5748322-abad-483c-ba35-10e1178e5a4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:51:25.037207 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.037181    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-certs" pod "f5748322-abad-483c-ba35-10e1178e5a4d" (UID: "f5748322-abad-483c-ba35-10e1178e5a4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:51:25.037312 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.037287    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5748322-abad-483c-ba35-10e1178e5a4d-istiod-ca-cert" pod "f5748322-abad-483c-ba35-10e1178e5a4d" (UID: "f5748322-abad-483c-ba35-10e1178e5a4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:51:25.037576 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.037552    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:51:25.037669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.037584    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5748322-abad-483c-ba35-10e1178e5a4d-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:51:25.037669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.037599    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:51:25.037669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.037615    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:51:25.037669 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.037629    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:51:25.039021 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.038999    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-kube-api-access-xp26s" pod "f5748322-abad-483c-ba35-10e1178e5a4d" (UID: "f5748322-abad-483c-ba35-10e1178e5a4d"). InnerVolumeSpecName "kube-api-access-xp26s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:51:25.039215 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.039188    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5748322-abad-483c-ba35-10e1178e5a4d-istio-podinfo" pod "f5748322-abad-483c-ba35-10e1178e5a4d" (UID: "f5748322-abad-483c-ba35-10e1178e5a4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:51:25.039488 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.039472    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-envoy" pod "f5748322-abad-483c-ba35-10e1178e5a4d" (UID: "f5748322-abad-483c-ba35-10e1178e5a4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:51:25.039639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.039620    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-istio-token" pod "f5748322-abad-483c-ba35-10e1178e5a4d" (UID: "f5748322-abad-483c-ba35-10e1178e5a4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:51:25.138774 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.138720    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5748322-abad-483c-ba35-10e1178e5a4d-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:51:25.138774 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.138775    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:51:25.138983 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.138790    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5748322-abad-483c-ba35-10e1178e5a4d-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:51:25.138983 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.138805    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp26s\" (UniqueName: \"kubernetes.io/projected/f5748322-abad-483c-ba35-10e1178e5a4d-kube-api-access-xp26s\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:51:25.657023 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.656988    5605 generic.go:356] "Generic (PLEG): container finished" podID="f5748322-abad-483c-ba35-10e1178e5a4d" containerID="5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4" exitCode=0
Nov 03 16:51:25.657583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.657038    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"
Nov 03 16:51:25.657583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.657076    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" event={"ID":"f5748322-abad-483c-ba35-10e1178e5a4d","Type":"ContainerDied","Data":"5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4"}
Nov 03 16:51:25.657583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.657133    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j" event={"ID":"f5748322-abad-483c-ba35-10e1178e5a4d","Type":"ContainerDied","Data":"7f8b4bcb2b4ad06330d8a763ed2dc14bc1f027a12e00593acd76c49c62da75b4"}
Nov 03 16:51:25.657583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.657160    5605 scope.go:122] "RemoveContainer" containerID="5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4"
Nov 03 16:51:25.667655 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.667633    5605 scope.go:122] "RemoveContainer" containerID="5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4"
Nov 03 16:51:25.667976 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:51:25.667951    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4\": container with ID starting with 5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4 not found: ID does not exist" containerID="5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4"
Nov 03 16:51:25.668068 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.667988    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4"} err="failed to get container status \"5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4\": rpc error: code = NotFound desc = could not find container \"5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4\": container with ID starting with 5b2a0d41d2aa50014570aea659042e1be8c0cffb0e7cf0276c3115a5e110ece4 not found: ID does not exist"
Nov 03 16:51:25.831522 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.831486    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"]
Nov 03 16:51:25.838056 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:25.838030    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-192564-openshift-default-6d8f48dd8f-jlk2j"]
Nov 03 16:51:26.309438 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:26.309402    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" path="/var/lib/kubelet/pods/f5748322-abad-483c-ba35-10e1178e5a4d/volumes"
Nov 03 16:51:28.650985 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:28.650910    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:51:33.393406 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:51:33.393366    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:51:54.645173 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:54.645147    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:51:54.662024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:54.661999    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:51:55.535569 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:51:55.535539    5605 scope.go:122] "RemoveContainer" containerID="bb20d2b5ddb5bb171d1b6c12f52062c735a805e465bbce6b55f4b3aa458d60b2"
Nov 03 16:52:03.394578 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:52:03.394539    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:52:33.393802 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:52:33.393758    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:52:39.160586 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.160544    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"]
Nov 03 16:52:39.161302 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.161285    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy"
Nov 03 16:52:39.161364 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.161305    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy"
Nov 03 16:52:39.161443 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.161431    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5748322-abad-483c-ba35-10e1178e5a4d" containerName="istio-proxy"
Nov 03 16:52:39.164860 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.164834    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.256798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.247973    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-963a2f-openshift-default-dockercfg-9n7j8\""
Nov 03 16:52:39.256798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.251596    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-certs\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.256798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.251645    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-istio-token\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.256798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.251680    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-envoy\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.256798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.251746    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4f6w\" (UniqueName: \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-kube-api-access-f4f6w\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.256798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.251789    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/033226c0-d209-48ae-9fef-205e34257ad3-istio-podinfo\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.256798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.251863    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-socket\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.256798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.252012    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/033226c0-d209-48ae-9fef-205e34257ad3-istiod-ca-cert\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.256798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.252103    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-data\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.256798 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.252151    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-credential-socket\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.290770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.290695    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"]
Nov 03 16:52:39.353471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.353418    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-data\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.353471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.353471    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-credential-socket\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.353787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.353501    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-certs\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.353787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.353517    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-istio-token\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.353787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.353543    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-envoy\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.353967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.353885    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4f6w\" (UniqueName: \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-kube-api-access-f4f6w\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.353967 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.353929    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/033226c0-d209-48ae-9fef-205e34257ad3-istio-podinfo\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.354074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.353964    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-certs\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.354074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.354000    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-data\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.354074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.354006    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-socket\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.354248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.354077    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-credential-socket\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.354248 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.354196    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/033226c0-d209-48ae-9fef-205e34257ad3-istiod-ca-cert\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.354370 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.354263    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-socket\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.354748 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.354700    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/033226c0-d209-48ae-9fef-205e34257ad3-istiod-ca-cert\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.356607 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.356585    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-envoy\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.362271 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.362245    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/033226c0-d209-48ae-9fef-205e34257ad3-istio-podinfo\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.417458 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.417395    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-istio-token\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.417795 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.417771    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4f6w\" (UniqueName: \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-kube-api-access-f4f6w\") pod \"e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") " pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.515033 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.514989    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:39.958344 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.958317    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"]
Nov 03 16:52:39.960103 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:52:39.960068    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod033226c0_d209_48ae_9fef_205e34257ad3.slice/crio-af3a0ca5e098aa6442e419f72b5cc504dffa3bd1c3bf82025efde3e65d4444e2 WatchSource:0}: Error finding container af3a0ca5e098aa6442e419f72b5cc504dffa3bd1c3bf82025efde3e65d4444e2: Status 404 returned error can't find the container with id af3a0ca5e098aa6442e419f72b5cc504dffa3bd1c3bf82025efde3e65d4444e2
Nov 03 16:52:39.962500 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.962468    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:52:39.962569 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.962549    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:52:39.962614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:39.962579    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:52:40.093130 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:40.093089    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" event={"ID":"033226c0-d209-48ae-9fef-205e34257ad3","Type":"ContainerStarted","Data":"df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e"}
Nov 03 16:52:40.093466 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:40.093161    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" event={"ID":"033226c0-d209-48ae-9fef-205e34257ad3","Type":"ContainerStarted","Data":"af3a0ca5e098aa6442e419f72b5cc504dffa3bd1c3bf82025efde3e65d4444e2"}
Nov 03 16:52:40.218490 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:40.218381    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" podStartSLOduration=1.218366066 podStartE2EDuration="1.218366066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:52:40.21638935 +0000 UTC m=+2446.040840621" watchObservedRunningTime="2026-11-03 16:52:40.218366066 +0000 UTC m=+2446.042817336"
Nov 03 16:52:40.515138 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:40.515050    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:41.521610 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:41.521575    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:42.104107 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:42.104078    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:42.105112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:42.105093    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:52:47.062788 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.060516    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"]
Nov 03 16:52:47.084262 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.084231    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 16:52:47.129227 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.129194    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"]
Nov 03 16:52:47.158528 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.158491    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-963a2f\""
Nov 03 16:52:47.158710 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.158530    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-963a2f-dockercfg-8f9qq\""
Nov 03 16:52:47.161583 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.161552    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf\" (UID: \"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44\") " pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 16:52:47.161706 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.161604    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b449\" (UniqueName: \"kubernetes.io/projected/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-kube-api-access-8b449\") pod \"payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf\" (UID: \"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44\") " pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 16:52:47.263407 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.263374    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf\" (UID: \"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44\") " pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 16:52:47.263587 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.263412    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b449\" (UniqueName: \"kubernetes.io/projected/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-kube-api-access-8b449\") pod \"payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf\" (UID: \"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44\") " pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 16:52:47.264181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.264154    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf\" (UID: \"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44\") " pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 16:52:47.335381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.335352    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b449\" (UniqueName: \"kubernetes.io/projected/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-kube-api-access-8b449\") pod \"payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf\" (UID: \"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44\") " pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 16:52:47.402995 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.402968    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 16:52:47.784502 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:47.784480    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"]
Nov 03 16:52:47.786541 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:52:47.786513    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d32ce6_2fd8_49b4_8ade_ab2c12b24e44.slice/crio-019d541dde5058ba3e722a014b65727d5d481509432d0a6d307be495d27d7b92 WatchSource:0}: Error finding container 019d541dde5058ba3e722a014b65727d5d481509432d0a6d307be495d27d7b92: Status 404 returned error can't find the container with id 019d541dde5058ba3e722a014b65727d5d481509432d0a6d307be495d27d7b92
Nov 03 16:52:48.148099 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:48.148063    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf" event={"ID":"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44","Type":"ContainerStarted","Data":"8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d"}
Nov 03 16:52:48.148507 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:48.148141    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf" event={"ID":"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44","Type":"ContainerStarted","Data":"019d541dde5058ba3e722a014b65727d5d481509432d0a6d307be495d27d7b92"}
Nov 03 16:52:48.148507 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:48.148210    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 16:52:48.269339 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:48.269281    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf" podStartSLOduration=1.269266133 podStartE2EDuration="1.269266133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:52:48.268412508 +0000 UTC m=+2454.092863779" watchObservedRunningTime="2026-11-03 16:52:48.269266133 +0000 UTC m=+2454.093717401"
Nov 03 16:52:59.157366 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:52:59.157284    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 16:53:03.393495 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:53:03.393457    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:53:33.393409 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:53:33.393366    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:54:03.393642 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:54:03.393599    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:54:15.748263 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:15.748229    5605 generic.go:356] "Generic (PLEG): container finished" podID="033226c0-d209-48ae-9fef-205e34257ad3" containerID="df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e" exitCode=0
Nov 03 16:54:15.748658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:15.748298    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" event={"ID":"033226c0-d209-48ae-9fef-205e34257ad3","Type":"ContainerDied","Data":"df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e"}
Nov 03 16:54:15.748713 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:15.748698    5605 scope.go:122] "RemoveContainer" containerID="df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e"
Nov 03 16:54:15.750241 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:15.750207    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:54:15.750336 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:15.750279    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:54:15.750336 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:15.750312    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 16:54:16.755558 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:16.755520    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" event={"ID":"033226c0-d209-48ae-9fef-205e34257ad3","Type":"ContainerStarted","Data":"82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13"}
Nov 03 16:54:17.515897 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:17.515850    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:54:17.516124 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:17.516103    5605 patch_prober.go:29] interesting pod/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Nov 03 16:54:17.516170 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:17.516148    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Nov 03 16:54:18.515501 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:18.515462    5605 patch_prober.go:29] interesting pod/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Nov 03 16:54:18.515953 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:18.515518    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Nov 03 16:54:19.515850 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:19.515817    5605 patch_prober.go:29] interesting pod/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Nov 03 16:54:19.516277 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:19.515869    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Nov 03 16:54:20.515887 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:20.515850    5605 patch_prober.go:29] interesting pod/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Nov 03 16:54:20.516292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:20.515911    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Nov 03 16:54:21.516288 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:21.516255    5605 patch_prober.go:29] interesting pod/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Nov 03 16:54:21.516690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:21.516309    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Nov 03 16:54:22.520260 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:22.520230    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:54:22.520799 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:22.520615    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:54:22.521385 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:22.521365    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 16:54:33.392317 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:54:33.392221    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:54:49.192794 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.192749    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"]
Nov 03 16:54:49.196082 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.196063    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.253594 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.253563    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"]
Nov 03 16:54:49.301810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.301782    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7e55c3\"/\"openshift-service-ca.crt\""
Nov 03 16:54:49.301810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.301791    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7e55c3\"/\"default-dockercfg-wqlq8\""
Nov 03 16:54:49.302428 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.301869    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7e55c3\"/\"test-model-7e55c3-kserve-self-signed-certs\""
Nov 03 16:54:49.302428 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.302107    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7e55c3\"/\"kube-root-ca.crt\""
Nov 03 16:54:49.308696 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.308676    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-tmp-dir\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.308827 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.308812    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-model-cache\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.308875 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.308867    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sncbt\" (UniqueName: \"kubernetes.io/projected/2dc3e0cc-1f59-4681-906a-4cb13741e308-kube-api-access-sncbt\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.308919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.308895    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dc3e0cc-1f59-4681-906a-4cb13741e308-tls-certs\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.308919 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.308915    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-home\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.309068 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.309052    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-dshm\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.410250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.410215    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-model-cache\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.410462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.410283    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sncbt\" (UniqueName: \"kubernetes.io/projected/2dc3e0cc-1f59-4681-906a-4cb13741e308-kube-api-access-sncbt\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.410462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.410312    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dc3e0cc-1f59-4681-906a-4cb13741e308-tls-certs\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.410462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.410332    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-home\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.410462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.410383    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-dshm\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.410462 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.410427    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-tmp-dir\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.410791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.410667    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-model-cache\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.410791 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.410776    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-home\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.410864 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.410856    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-tmp-dir\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.412852 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.412826    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-dshm\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.413084 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.413068    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dc3e0cc-1f59-4681-906a-4cb13741e308-tls-certs\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.471089 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.471024    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sncbt\" (UniqueName: \"kubernetes.io/projected/2dc3e0cc-1f59-4681-906a-4cb13741e308-kube-api-access-sncbt\") pod \"test-model-7e55c3-kserve-768966df8c-b2w5c\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") " pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.508149 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.508117    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:49.901694 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.901668    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"]
Nov 03 16:54:49.902694 ip-10-0-142-22 kubenswrapper[5605]: W1103 16:54:49.902669    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dc3e0cc_1f59_4681_906a_4cb13741e308.slice/crio-21a6fb5c6823e1f6a888a547a4953d81db6e6a7094311a93d13e2c80f4e29d43 WatchSource:0}: Error finding container 21a6fb5c6823e1f6a888a547a4953d81db6e6a7094311a93d13e2c80f4e29d43: Status 404 returned error can't find the container with id 21a6fb5c6823e1f6a888a547a4953d81db6e6a7094311a93d13e2c80f4e29d43
Nov 03 16:54:49.958446 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:49.958417    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c" event={"ID":"2dc3e0cc-1f59-4681-906a-4cb13741e308","Type":"ContainerStarted","Data":"21a6fb5c6823e1f6a888a547a4953d81db6e6a7094311a93d13e2c80f4e29d43"}
Nov 03 16:54:50.965913 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:50.965876    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c" event={"ID":"2dc3e0cc-1f59-4681-906a-4cb13741e308","Type":"ContainerStarted","Data":"eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7"}
Nov 03 16:54:51.089276 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:51.089225    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c" podStartSLOduration=2.08921168 podStartE2EDuration="2.08921168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 16:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 16:54:51.088800501 +0000 UTC m=+2576.913251771" watchObservedRunningTime="2026-11-03 16:54:51.08921168 +0000 UTC m=+2576.913662983"
Nov 03 16:54:51.509214 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:51.509168    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:59.508575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:59.508542    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:54:59.522014 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:54:59.521978    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:55:00.050253 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:00.050224    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:55:03.393944 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:55:03.393901    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:55:33.392698 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:55:33.392662    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:55:46.749965 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:46.749545    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"]
Nov 03 16:55:46.749965 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:46.749833    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy" containerID="cri-o://8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0" gracePeriod=30
Nov 03 16:55:46.777481 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:46.777451    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"]
Nov 03 16:55:46.777781 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:46.777710    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy" containerID="cri-o://40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf" gracePeriod=30
Nov 03 16:55:46.780772 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:46.780701    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"]
Nov 03 16:55:46.781022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:46.780954    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy" containerID="cri-o://ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f" gracePeriod=30
Nov 03 16:55:52.075100 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.075061    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:55:52.105097 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.105071    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:55:52.108377 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.108352    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:55:52.114009 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.113988    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-certs\") pod \"1b444a3c-d720-476d-81b9-b4e15407c2e3\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") "
Nov 03 16:55:52.114105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114073    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-podinfo\") pod \"1b444a3c-d720-476d-81b9-b4e15407c2e3\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") "
Nov 03 16:55:52.114105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114101    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-envoy\") pod \"1b444a3c-d720-476d-81b9-b4e15407c2e3\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") "
Nov 03 16:55:52.114184 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114153    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-credential-socket\") pod \"1b444a3c-d720-476d-81b9-b4e15407c2e3\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") "
Nov 03 16:55:52.114225 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114216    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-data\") pod \"1b444a3c-d720-476d-81b9-b4e15407c2e3\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") "
Nov 03 16:55:52.114225 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114214    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-certs" pod "1b444a3c-d720-476d-81b9-b4e15407c2e3" (UID: "1b444a3c-d720-476d-81b9-b4e15407c2e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.114331 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114280    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b444a3c-d720-476d-81b9-b4e15407c2e3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b444a3c-d720-476d-81b9-b4e15407c2e3-istiod-ca-cert\") pod \"1b444a3c-d720-476d-81b9-b4e15407c2e3\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") "
Nov 03 16:55:52.114391 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114338    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-socket\") pod \"1b444a3c-d720-476d-81b9-b4e15407c2e3\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") "
Nov 03 16:55:52.114447 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114386    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-token\" (UniqueName: \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-token\") pod \"1b444a3c-d720-476d-81b9-b4e15407c2e3\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") "
Nov 03 16:55:52.114447 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114420    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-kube-api-access-lwxg7\" (UniqueName: \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-kube-api-access-lwxg7\") pod \"1b444a3c-d720-476d-81b9-b4e15407c2e3\" (UID: \"1b444a3c-d720-476d-81b9-b4e15407c2e3\") "
Nov 03 16:55:52.114551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114469    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-credential-socket" pod "1b444a3c-d720-476d-81b9-b4e15407c2e3" (UID: "1b444a3c-d720-476d-81b9-b4e15407c2e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.114551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114486    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-data" pod "1b444a3c-d720-476d-81b9-b4e15407c2e3" (UID: "1b444a3c-d720-476d-81b9-b4e15407c2e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.114639 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114609    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-socket" pod "1b444a3c-d720-476d-81b9-b4e15407c2e3" (UID: "1b444a3c-d720-476d-81b9-b4e15407c2e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.114977 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114955    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b444a3c-d720-476d-81b9-b4e15407c2e3-istiod-ca-cert" pod "1b444a3c-d720-476d-81b9-b4e15407c2e3" (UID: "1b444a3c-d720-476d-81b9-b4e15407c2e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:55:52.114977 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114958    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.115159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.114999    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.115159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.115015    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.115159 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.115029    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.117058 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.117038    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-envoy" pod "1b444a3c-d720-476d-81b9-b4e15407c2e3" (UID: "1b444a3c-d720-476d-81b9-b4e15407c2e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.117058 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.117041    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-podinfo" pod "1b444a3c-d720-476d-81b9-b4e15407c2e3" (UID: "1b444a3c-d720-476d-81b9-b4e15407c2e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:55:52.117292 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.117272    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-kube-api-access-lwxg7" pod "1b444a3c-d720-476d-81b9-b4e15407c2e3" (UID: "1b444a3c-d720-476d-81b9-b4e15407c2e3"). InnerVolumeSpecName "kube-api-access-lwxg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:55:52.117364 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.117306    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-token" pod "1b444a3c-d720-476d-81b9-b4e15407c2e3" (UID: "1b444a3c-d720-476d-81b9-b4e15407c2e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:55:52.215975 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.215941    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-kube-api-access-btlmp\" (UniqueName: \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-kube-api-access-btlmp\") pod \"ca21abb4-1609-4b5e-81b8-72cdc2481417\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") "
Nov 03 16:55:52.216181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.215986    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-envoy\") pod \"ca21abb4-1609-4b5e-81b8-72cdc2481417\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") "
Nov 03 16:55:52.216181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216026    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca21abb4-1609-4b5e-81b8-72cdc2481417-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca21abb4-1609-4b5e-81b8-72cdc2481417-istiod-ca-cert\") pod \"ca21abb4-1609-4b5e-81b8-72cdc2481417\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") "
Nov 03 16:55:52.216181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216074    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-credential-socket\") pod \"ca21abb4-1609-4b5e-81b8-72cdc2481417\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") "
Nov 03 16:55:52.216181 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216111    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9c549fd-9a32-4a08-a3ce-bf698639f434-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9c549fd-9a32-4a08-a3ce-bf698639f434-istiod-ca-cert\") pod \"a9c549fd-9a32-4a08-a3ce-bf698639f434\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") "
Nov 03 16:55:52.216393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216191    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-podinfo\") pod \"a9c549fd-9a32-4a08-a3ce-bf698639f434\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") "
Nov 03 16:55:52.216393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216230    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-credential-socket\") pod \"a9c549fd-9a32-4a08-a3ce-bf698639f434\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") "
Nov 03 16:55:52.216393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216327    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-socket\") pod \"a9c549fd-9a32-4a08-a3ce-bf698639f434\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") "
Nov 03 16:55:52.216393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216361    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-socket\") pod \"ca21abb4-1609-4b5e-81b8-72cdc2481417\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") "
Nov 03 16:55:52.216393 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216390    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-token\" (UniqueName: \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-token\") pod \"a9c549fd-9a32-4a08-a3ce-bf698639f434\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") "
Nov 03 16:55:52.216662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216407    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-credential-socket" pod "ca21abb4-1609-4b5e-81b8-72cdc2481417" (UID: "ca21abb4-1609-4b5e-81b8-72cdc2481417"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.216662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216470    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-certs\") pod \"ca21abb4-1609-4b5e-81b8-72cdc2481417\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") "
Nov 03 16:55:52.216662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216492    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca21abb4-1609-4b5e-81b8-72cdc2481417-istiod-ca-cert" pod "ca21abb4-1609-4b5e-81b8-72cdc2481417" (UID: "ca21abb4-1609-4b5e-81b8-72cdc2481417"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:55:52.216662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216518    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-podinfo\") pod \"ca21abb4-1609-4b5e-81b8-72cdc2481417\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") "
Nov 03 16:55:52.216662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216552    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-envoy\") pod \"a9c549fd-9a32-4a08-a3ce-bf698639f434\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") "
Nov 03 16:55:52.216662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216564    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9c549fd-9a32-4a08-a3ce-bf698639f434-istiod-ca-cert" pod "a9c549fd-9a32-4a08-a3ce-bf698639f434" (UID: "a9c549fd-9a32-4a08-a3ce-bf698639f434"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:55:52.216662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216603    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-data\") pod \"a9c549fd-9a32-4a08-a3ce-bf698639f434\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") "
Nov 03 16:55:52.217093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216674    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-data\") pod \"ca21abb4-1609-4b5e-81b8-72cdc2481417\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") "
Nov 03 16:55:52.217093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216713    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-certs\") pod \"a9c549fd-9a32-4a08-a3ce-bf698639f434\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") "
Nov 03 16:55:52.217093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216765    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-kube-api-access-7twl7\" (UniqueName: \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-kube-api-access-7twl7\") pod \"a9c549fd-9a32-4a08-a3ce-bf698639f434\" (UID: \"a9c549fd-9a32-4a08-a3ce-bf698639f434\") "
Nov 03 16:55:52.217093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216849    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-socket" pod "a9c549fd-9a32-4a08-a3ce-bf698639f434" (UID: "a9c549fd-9a32-4a08-a3ce-bf698639f434"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.217093 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.216958    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-token\" (UniqueName: \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-token\") pod \"ca21abb4-1609-4b5e-81b8-72cdc2481417\" (UID: \"ca21abb4-1609-4b5e-81b8-72cdc2481417\") "
Nov 03 16:55:52.217357 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217340    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-data" pod "a9c549fd-9a32-4a08-a3ce-bf698639f434" (UID: "a9c549fd-9a32-4a08-a3ce-bf698639f434"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.217496 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217471    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-credential-socket" pod "a9c549fd-9a32-4a08-a3ce-bf698639f434" (UID: "a9c549fd-9a32-4a08-a3ce-bf698639f434"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.217662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217615    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.217662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217641    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.217662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217657    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.218074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217672    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca21abb4-1609-4b5e-81b8-72cdc2481417-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.218074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217690    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.218074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217705    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9c549fd-9a32-4a08-a3ce-bf698639f434-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.218074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217719    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b444a3c-d720-476d-81b9-b4e15407c2e3-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.218074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217756    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.218074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217770    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.218074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217785    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.218074 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.217801    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwxg7\" (UniqueName: \"kubernetes.io/projected/1b444a3c-d720-476d-81b9-b4e15407c2e3-kube-api-access-lwxg7\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.218826 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.218227    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-certs" pod "a9c549fd-9a32-4a08-a3ce-bf698639f434" (UID: "a9c549fd-9a32-4a08-a3ce-bf698639f434"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.218826 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.218431    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-certs" pod "ca21abb4-1609-4b5e-81b8-72cdc2481417" (UID: "ca21abb4-1609-4b5e-81b8-72cdc2481417"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.218826 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.218691    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-socket" pod "ca21abb4-1609-4b5e-81b8-72cdc2481417" (UID: "ca21abb4-1609-4b5e-81b8-72cdc2481417"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.218826 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.218789    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-data" pod "ca21abb4-1609-4b5e-81b8-72cdc2481417" (UID: "ca21abb4-1609-4b5e-81b8-72cdc2481417"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.219452 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.219061    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-kube-api-access-btlmp" pod "ca21abb4-1609-4b5e-81b8-72cdc2481417" (UID: "ca21abb4-1609-4b5e-81b8-72cdc2481417"). InnerVolumeSpecName "kube-api-access-btlmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:55:52.219452 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.219266    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-podinfo" pod "a9c549fd-9a32-4a08-a3ce-bf698639f434" (UID: "a9c549fd-9a32-4a08-a3ce-bf698639f434"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:55:52.219580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.219528    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-envoy" pod "ca21abb4-1609-4b5e-81b8-72cdc2481417" (UID: "ca21abb4-1609-4b5e-81b8-72cdc2481417"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.219873 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.219849    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-envoy" pod "a9c549fd-9a32-4a08-a3ce-bf698639f434" (UID: "a9c549fd-9a32-4a08-a3ce-bf698639f434"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:55:52.220033 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.220001    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-token" pod "ca21abb4-1609-4b5e-81b8-72cdc2481417" (UID: "ca21abb4-1609-4b5e-81b8-72cdc2481417"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:55:52.220165 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.220147    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-token" pod "a9c549fd-9a32-4a08-a3ce-bf698639f434" (UID: "a9c549fd-9a32-4a08-a3ce-bf698639f434"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:55:52.220688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.220669    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-kube-api-access-7twl7" pod "a9c549fd-9a32-4a08-a3ce-bf698639f434" (UID: "a9c549fd-9a32-4a08-a3ce-bf698639f434"). InnerVolumeSpecName "kube-api-access-7twl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:55:52.220828 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.220810    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-podinfo" pod "ca21abb4-1609-4b5e-81b8-72cdc2481417" (UID: "ca21abb4-1609-4b5e-81b8-72cdc2481417"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:55:52.318720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318649    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.318720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318680    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.318720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318690    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7twl7\" (UniqueName: \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-kube-api-access-7twl7\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.318720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318701    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.318720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318710    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btlmp\" (UniqueName: \"kubernetes.io/projected/ca21abb4-1609-4b5e-81b8-72cdc2481417-kube-api-access-btlmp\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.318720 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318718    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.319166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318754    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.319166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318767    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.319166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318776    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.319166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318784    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca21abb4-1609-4b5e-81b8-72cdc2481417-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.319166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318793    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca21abb4-1609-4b5e-81b8-72cdc2481417-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.319166 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.318817    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9c549fd-9a32-4a08-a3ce-bf698639f434-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:55:52.355105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.355060    5605 generic.go:356] "Generic (PLEG): container finished" podID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerID="8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0" exitCode=0
Nov 03 16:55:52.355285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.355114    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"
Nov 03 16:55:52.355285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.355137    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" event={"ID":"1b444a3c-d720-476d-81b9-b4e15407c2e3","Type":"ContainerDied","Data":"8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0"}
Nov 03 16:55:52.355285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.355196    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf" event={"ID":"1b444a3c-d720-476d-81b9-b4e15407c2e3","Type":"ContainerDied","Data":"eece10c78f407c9464da2a6b0164a7837ab3a9e2e61e8d02de3cc34a11b75bf0"}
Nov 03 16:55:52.355285 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.355216    5605 scope.go:122] "RemoveContainer" containerID="8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0"
Nov 03 16:55:52.357397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.356703    5605 generic.go:356] "Generic (PLEG): container finished" podID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerID="ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f" exitCode=0
Nov 03 16:55:52.357397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.356862    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"
Nov 03 16:55:52.357397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.357065    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" event={"ID":"a9c549fd-9a32-4a08-a3ce-bf698639f434","Type":"ContainerDied","Data":"ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f"}
Nov 03 16:55:52.357397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.357109    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq" event={"ID":"a9c549fd-9a32-4a08-a3ce-bf698639f434","Type":"ContainerDied","Data":"d9945a6fde42839ec30c78305c73e34d023169a31af3d92fabb1ffae0eda8e23"}
Nov 03 16:55:52.359122 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.359098    5605 generic.go:356] "Generic (PLEG): container finished" podID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerID="40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf" exitCode=0
Nov 03 16:55:52.359246 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.359137    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"
Nov 03 16:55:52.359246 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.359196    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" event={"ID":"ca21abb4-1609-4b5e-81b8-72cdc2481417","Type":"ContainerDied","Data":"40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf"}
Nov 03 16:55:52.359246 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.359241    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb" event={"ID":"ca21abb4-1609-4b5e-81b8-72cdc2481417","Type":"ContainerDied","Data":"f421c7683b6af7034c1b25dbcd03a67eeb4ed5b0e10f1509010930021f3eddd1"}
Nov 03 16:55:52.389520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.381982    5605 scope.go:122] "RemoveContainer" containerID="8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0"
Nov 03 16:55:52.396963 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:55:52.396917    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0\": container with ID starting with 8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0 not found: ID does not exist" containerID="8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0"
Nov 03 16:55:52.397158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.396997    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0"} err="failed to get container status \"8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0\": rpc error: code = NotFound desc = could not find container \"8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0\": container with ID starting with 8cadfcd2cac5ae4499de73171c81a388cdba473a7b1066d5d8072b0c23cc5cc0 not found: ID does not exist"
Nov 03 16:55:52.397158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.397026    5605 scope.go:122] "RemoveContainer" containerID="ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f"
Nov 03 16:55:52.416141 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.416116    5605 scope.go:122] "RemoveContainer" containerID="ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f"
Nov 03 16:55:52.416540 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:55:52.416510    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f\": container with ID starting with ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f not found: ID does not exist" containerID="ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f"
Nov 03 16:55:52.416656 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.416551    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f"} err="failed to get container status \"ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f\": rpc error: code = NotFound desc = could not find container \"ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f\": container with ID starting with ca17f38a88c30e37759389a8a170f1a078d43192b56b407f8c60609cfce5de5f not found: ID does not exist"
Nov 03 16:55:52.416656 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.416577    5605 scope.go:122] "RemoveContainer" containerID="40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf"
Nov 03 16:55:52.426616 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.426599    5605 scope.go:122] "RemoveContainer" containerID="40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf"
Nov 03 16:55:52.427003 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:55:52.426983    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf\": container with ID starting with 40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf not found: ID does not exist" containerID="40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf"
Nov 03 16:55:52.427078 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.427011    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf"} err="failed to get container status \"40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf\": rpc error: code = NotFound desc = could not find container \"40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf\": container with ID starting with 40536c5635b102cad8486ed2dcbb8c49c1f6e4aaa7ecff4a29f1fbe4c21427bf not found: ID does not exist"
Nov 03 16:55:52.630303 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.630273    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"]
Nov 03 16:55:52.638585 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.638555    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cvpvf"]
Nov 03 16:55:52.752291 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.752258    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"]
Nov 03 16:55:52.760688 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.760660    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-xfzlb"]
Nov 03 16:55:52.875051 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.875016    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"]
Nov 03 16:55:52.880770 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:52.880686    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mfgcq"]
Nov 03 16:55:54.314790 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:54.314761    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" path="/var/lib/kubelet/pods/1b444a3c-d720-476d-81b9-b4e15407c2e3/volumes"
Nov 03 16:55:54.315250 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:54.315235    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" path="/var/lib/kubelet/pods/a9c549fd-9a32-4a08-a3ce-bf698639f434/volumes"
Nov 03 16:55:54.315672 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:54.315658    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" path="/var/lib/kubelet/pods/ca21abb4-1609-4b5e-81b8-72cdc2481417/volumes"
Nov 03 16:55:56.052622 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:56.052533    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"]
Nov 03 16:55:56.053041 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:56.052767    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy" containerID="cri-o://05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c" gracePeriod=30
Nov 03 16:55:56.168588 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:56.168556    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"]
Nov 03 16:55:56.168862 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:55:56.168821    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy" containerID="cri-o://b9fa72bfae38a8b4e3b1aafa3c5a5f460b3284d6cbd3752c2565b9b21f1e814b" gracePeriod=30
Nov 03 16:56:01.319892 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.319804    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:56:01.418520 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.418492    5605 generic.go:356] "Generic (PLEG): container finished" podID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerID="b9fa72bfae38a8b4e3b1aafa3c5a5f460b3284d6cbd3752c2565b9b21f1e814b" exitCode=0
Nov 03 16:56:01.418662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.418561    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" event={"ID":"bfd53f66-eba2-4ad7-a908-a42c9b6914d9","Type":"ContainerDied","Data":"b9fa72bfae38a8b4e3b1aafa3c5a5f460b3284d6cbd3752c2565b9b21f1e814b"}
Nov 03 16:56:01.418662 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.418621    5605 scope.go:122] "RemoveContainer" containerID="e2c9c2e15e45c9815a9b258c622cba8c360020710b63b1a319dec8d5e4498a93"
Nov 03 16:56:01.420614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.420592    5605 generic.go:356] "Generic (PLEG): container finished" podID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerID="05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c" exitCode=0
Nov 03 16:56:01.420754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.420637    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" event={"ID":"ae2b9d64-981e-419f-84be-3beae332c9d1","Type":"ContainerDied","Data":"05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c"}
Nov 03 16:56:01.420754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.420642    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"
Nov 03 16:56:01.420754 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.420667    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh" event={"ID":"ae2b9d64-981e-419f-84be-3beae332c9d1","Type":"ContainerDied","Data":"e3f73ae9cc169a4594331b039b3780410ce10753f8e1a136097032f5f47fd6d1"}
Nov 03 16:56:01.429008 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.428986    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-certs\") pod \"ae2b9d64-981e-419f-84be-3beae332c9d1\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") "
Nov 03 16:56:01.429112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429043    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-data\") pod \"ae2b9d64-981e-419f-84be-3beae332c9d1\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") "
Nov 03 16:56:01.429112 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429075    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-podinfo\") pod \"ae2b9d64-981e-419f-84be-3beae332c9d1\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") "
Nov 03 16:56:01.429254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429117    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-kube-api-access-2765h\" (UniqueName: \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-kube-api-access-2765h\") pod \"ae2b9d64-981e-419f-84be-3beae332c9d1\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") "
Nov 03 16:56:01.429254 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429192    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-envoy\") pod \"ae2b9d64-981e-419f-84be-3beae332c9d1\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") "
Nov 03 16:56:01.429369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429288    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-data" pod "ae2b9d64-981e-419f-84be-3beae332c9d1" (UID: "ae2b9d64-981e-419f-84be-3beae332c9d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:56:01.429369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429300    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-token\" (UniqueName: \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-token\") pod \"ae2b9d64-981e-419f-84be-3beae332c9d1\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") "
Nov 03 16:56:01.429369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429303    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-certs" pod "ae2b9d64-981e-419f-84be-3beae332c9d1" (UID: "ae2b9d64-981e-419f-84be-3beae332c9d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:56:01.429369 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429347    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-credential-socket\") pod \"ae2b9d64-981e-419f-84be-3beae332c9d1\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") "
Nov 03 16:56:01.429575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429384    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae2b9d64-981e-419f-84be-3beae332c9d1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae2b9d64-981e-419f-84be-3beae332c9d1-istiod-ca-cert\") pod \"ae2b9d64-981e-419f-84be-3beae332c9d1\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") "
Nov 03 16:56:01.429575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429448    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-socket\") pod \"ae2b9d64-981e-419f-84be-3beae332c9d1\" (UID: \"ae2b9d64-981e-419f-84be-3beae332c9d1\") "
Nov 03 16:56:01.430210 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429763    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-credential-socket" pod "ae2b9d64-981e-419f-84be-3beae332c9d1" (UID: "ae2b9d64-981e-419f-84be-3beae332c9d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:56:01.430210 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429881    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae2b9d64-981e-419f-84be-3beae332c9d1-istiod-ca-cert" pod "ae2b9d64-981e-419f-84be-3beae332c9d1" (UID: "ae2b9d64-981e-419f-84be-3beae332c9d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:56:01.430210 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.429972    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-socket" pod "ae2b9d64-981e-419f-84be-3beae332c9d1" (UID: "ae2b9d64-981e-419f-84be-3beae332c9d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:56:01.430472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.430225    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.430472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.430244    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.430472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.430261    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae2b9d64-981e-419f-84be-3beae332c9d1-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.430472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.430276    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.430472 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.430292    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.431712 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.431679    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-envoy" pod "ae2b9d64-981e-419f-84be-3beae332c9d1" (UID: "ae2b9d64-981e-419f-84be-3beae332c9d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:56:01.431852 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.431824    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-podinfo" pod "ae2b9d64-981e-419f-84be-3beae332c9d1" (UID: "ae2b9d64-981e-419f-84be-3beae332c9d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:56:01.432134 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.432119    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-kube-api-access-2765h" pod "ae2b9d64-981e-419f-84be-3beae332c9d1" (UID: "ae2b9d64-981e-419f-84be-3beae332c9d1"). InnerVolumeSpecName "kube-api-access-2765h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:56:01.432498 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.432474    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-token" pod "ae2b9d64-981e-419f-84be-3beae332c9d1" (UID: "ae2b9d64-981e-419f-84be-3beae332c9d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:56:01.435955 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.435937    5605 scope.go:122] "RemoveContainer" containerID="05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c"
Nov 03 16:56:01.438295 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.438274    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:56:01.444797 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.444775    5605 scope.go:122] "RemoveContainer" containerID="6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1"
Nov 03 16:56:01.453623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.453595    5605 scope.go:122] "RemoveContainer" containerID="05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c"
Nov 03 16:56:01.453958 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:56:01.453932    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c\": container with ID starting with 05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c not found: ID does not exist" containerID="05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c"
Nov 03 16:56:01.454024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.453966    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c"} err="failed to get container status \"05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c\": rpc error: code = NotFound desc = could not find container \"05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c\": container with ID starting with 05750c31ce0ffb6381c8206e8d22b09c70221eb06689a6347baf6e619250313c not found: ID does not exist"
Nov 03 16:56:01.454024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.453984    5605 scope.go:122] "RemoveContainer" containerID="6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1"
Nov 03 16:56:01.454254 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:56:01.454236    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1\": container with ID starting with 6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1 not found: ID does not exist" containerID="6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1"
Nov 03 16:56:01.454295 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.454260    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1"} err="failed to get container status \"6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1\": rpc error: code = NotFound desc = could not find container \"6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1\": container with ID starting with 6d731554806941f2e8c2908472203f5bcf71e68ca807f432868245092b6f3aa1 not found: ID does not exist"
Nov 03 16:56:01.531395 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531358    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-kube-api-access-59h5v\" (UniqueName: \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-kube-api-access-59h5v\") pod \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") "
Nov 03 16:56:01.531575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531430    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-envoy\") pod \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") "
Nov 03 16:56:01.531575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531463    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-credential-socket\") pod \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") "
Nov 03 16:56:01.531575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531493    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-certs\") pod \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") "
Nov 03 16:56:01.531575 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531541    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istiod-ca-cert\") pod \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") "
Nov 03 16:56:01.531870 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531588    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-podinfo\") pod \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") "
Nov 03 16:56:01.531870 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531627    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-data\") pod \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") "
Nov 03 16:56:01.531870 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531684    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-token\" (UniqueName: \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-token\") pod \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") "
Nov 03 16:56:01.531870 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531716    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-socket\") pod \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\" (UID: \"bfd53f66-eba2-4ad7-a908-a42c9b6914d9\") "
Nov 03 16:56:01.531870 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531855    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-credential-socket" pod "bfd53f66-eba2-4ad7-a908-a42c9b6914d9" (UID: "bfd53f66-eba2-4ad7-a908-a42c9b6914d9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:56:01.532143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.531882    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-certs" pod "bfd53f66-eba2-4ad7-a908-a42c9b6914d9" (UID: "bfd53f66-eba2-4ad7-a908-a42c9b6914d9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:56:01.532143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532082    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istiod-ca-cert" pod "bfd53f66-eba2-4ad7-a908-a42c9b6914d9" (UID: "bfd53f66-eba2-4ad7-a908-a42c9b6914d9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:56:01.532417 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532204    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-socket" pod "bfd53f66-eba2-4ad7-a908-a42c9b6914d9" (UID: "bfd53f66-eba2-4ad7-a908-a42c9b6914d9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:56:01.532417 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532238    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-data" pod "bfd53f66-eba2-4ad7-a908-a42c9b6914d9" (UID: "bfd53f66-eba2-4ad7-a908-a42c9b6914d9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:56:01.532557 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532536    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.532626 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532563    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.532626 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532580    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2765h\" (UniqueName: \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-kube-api-access-2765h\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.532626 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532598    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.532626 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532613    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.532869 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532627    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae2b9d64-981e-419f-84be-3beae332c9d1-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.532869 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532641    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.532869 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532658    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.532869 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.532672    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.533956 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.533916    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-envoy" pod "bfd53f66-eba2-4ad7-a908-a42c9b6914d9" (UID: "bfd53f66-eba2-4ad7-a908-a42c9b6914d9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:56:01.534059 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.534029    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-token" pod "bfd53f66-eba2-4ad7-a908-a42c9b6914d9" (UID: "bfd53f66-eba2-4ad7-a908-a42c9b6914d9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:56:01.534129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.534069    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-kube-api-access-59h5v" pod "bfd53f66-eba2-4ad7-a908-a42c9b6914d9" (UID: "bfd53f66-eba2-4ad7-a908-a42c9b6914d9"). InnerVolumeSpecName "kube-api-access-59h5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:56:01.534176 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.534123    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-podinfo" pod "bfd53f66-eba2-4ad7-a908-a42c9b6914d9" (UID: "bfd53f66-eba2-4ad7-a908-a42c9b6914d9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:56:01.634376 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.634341    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.634376 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.634374    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.634376 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.634384    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59h5v\" (UniqueName: \"kubernetes.io/projected/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-kube-api-access-59h5v\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.634651 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.634396    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfd53f66-eba2-4ad7-a908-a42c9b6914d9-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:56:01.855587 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.855549    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"]
Nov 03 16:56:01.865963 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:01.865929    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c88d54-openshift-default-6d8d987d6f-xsxfh"]
Nov 03 16:56:02.309123 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:02.309090    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" path="/var/lib/kubelet/pods/ae2b9d64-981e-419f-84be-3beae332c9d1/volumes"
Nov 03 16:56:02.427787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:02.427760    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"
Nov 03 16:56:02.427787 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:02.427782    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf" event={"ID":"bfd53f66-eba2-4ad7-a908-a42c9b6914d9","Type":"ContainerDied","Data":"32b8f4df1bd59c9e2e8dcff79f851ffe834a232d855f93433791b8bd0732a266"}
Nov 03 16:56:02.428381 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:02.427833    5605 scope.go:122] "RemoveContainer" containerID="b9fa72bfae38a8b4e3b1aafa3c5a5f460b3284d6cbd3752c2565b9b21f1e814b"
Nov 03 16:56:02.669647 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:02.669612    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"]
Nov 03 16:56:02.682451 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:02.682388    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-42da0f-openshift-default-69bcdf78d7-8s2rf"]
Nov 03 16:56:03.392301 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:56:03.392264    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:56:04.308192 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:04.308162    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" path="/var/lib/kubelet/pods/bfd53f66-eba2-4ad7-a908-a42c9b6914d9/volumes"
Nov 03 16:56:33.393177 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:56:33.393139    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:56:54.681765 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:54.681706    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:56:54.699837 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:56:54.699812    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 16:57:03.392679 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:57:03.392634    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:57:33.391878 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:57:33.391800    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:58:03.392571 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:58:03.392534    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:58:07.988504 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:07.988456    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"]
Nov 03 16:58:08.099937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:08.099905    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"]
Nov 03 16:58:12.989045 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:12.988977    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd" podUID="f5c1f5f4-78bb-4305-9213-1660fcc23cbf" containerName="payload-processing" containerID="cri-o://bc0fe906f028f0ea58e437d60307b606047582e96bba0d8b37ffc80b91a2097a" gracePeriod=25
Nov 03 16:58:13.100958 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:13.100914    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x" podUID="29ac9424-33f5-48f0-83e2-022d1b036959" containerName="payload-pre-processing" containerID="cri-o://3e4b847b430db08147c4917c7e1e598a42c316ea688dc1f30f78a7b34854bbd3" gracePeriod=25
Nov 03 16:58:13.175179 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:13.175139    5605 generic.go:356] "Generic (PLEG): container finished" podID="f5c1f5f4-78bb-4305-9213-1660fcc23cbf" containerID="bc0fe906f028f0ea58e437d60307b606047582e96bba0d8b37ffc80b91a2097a" exitCode=0
Nov 03 16:58:13.175347 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:13.175211    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd" event={"ID":"f5c1f5f4-78bb-4305-9213-1660fcc23cbf","Type":"ContainerDied","Data":"bc0fe906f028f0ea58e437d60307b606047582e96bba0d8b37ffc80b91a2097a"}
Nov 03 16:58:13.244637 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:13.244572    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:58:13.339571 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:13.339544    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-plugins-config-volume\") pod \"f5c1f5f4-78bb-4305-9213-1660fcc23cbf\" (UID: \"f5c1f5f4-78bb-4305-9213-1660fcc23cbf\") "
Nov 03 16:58:13.339810 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:13.339628    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-kube-api-access-cn6lw\" (UniqueName: \"kubernetes.io/projected/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-kube-api-access-cn6lw\") pod \"f5c1f5f4-78bb-4305-9213-1660fcc23cbf\" (UID: \"f5c1f5f4-78bb-4305-9213-1660fcc23cbf\") "
Nov 03 16:58:13.339947 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:13.339923    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-plugins-config-volume" pod "f5c1f5f4-78bb-4305-9213-1660fcc23cbf" (UID: "f5c1f5f4-78bb-4305-9213-1660fcc23cbf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:58:13.340174 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:13.340156    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:13.341861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:13.341843    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-kube-api-access-cn6lw" pod "f5c1f5f4-78bb-4305-9213-1660fcc23cbf" (UID: "f5c1f5f4-78bb-4305-9213-1660fcc23cbf"). InnerVolumeSpecName "kube-api-access-cn6lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:58:13.441524 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:13.441493    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn6lw\" (UniqueName: \"kubernetes.io/projected/f5c1f5f4-78bb-4305-9213-1660fcc23cbf-kube-api-access-cn6lw\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:14.182327 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:14.182298    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"
Nov 03 16:58:14.182849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:14.182332    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd" event={"ID":"f5c1f5f4-78bb-4305-9213-1660fcc23cbf","Type":"ContainerDied","Data":"76feefd353834a18702d312f7a1762c829528f144c9507fa9c70dabacd3310e8"}
Nov 03 16:58:14.182849 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:14.182389    5605 scope.go:122] "RemoveContainer" containerID="bc0fe906f028f0ea58e437d60307b606047582e96bba0d8b37ffc80b91a2097a"
Nov 03 16:58:14.359158 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:14.359129    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"]
Nov 03 16:58:14.410343 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:14.410310    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4d9b-56b5c47d7c-w6fkd"]
Nov 03 16:58:16.309255 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:16.309226    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5c1f5f4-78bb-4305-9213-1660fcc23cbf" path="/var/lib/kubelet/pods/f5c1f5f4-78bb-4305-9213-1660fcc23cbf/volumes"
Nov 03 16:58:22.478217 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:22.478174    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x" podUID="29ac9424-33f5-48f0-83e2-022d1b036959" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 16:58:24.412433 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.412399    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"]
Nov 03 16:58:24.412861 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.412600    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx" podUID="f9cacf9e-deaf-41a7-87ad-7915154f4ac8" containerName="maas-api" containerID="cri-o://c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413" gracePeriod=30
Nov 03 16:58:24.692608 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.692584    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:58:24.881489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.881455    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-metrics-certs\") pod \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") "
Nov 03 16:58:24.881489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.881494    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-maas-api-tls\") pod \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") "
Nov 03 16:58:24.881766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.881568    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-openshift-service-ca\") pod \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") "
Nov 03 16:58:24.881766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.881702    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-kube-api-access-wzstk\" (UniqueName: \"kubernetes.io/projected/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-kube-api-access-wzstk\") pod \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\" (UID: \"f9cacf9e-deaf-41a7-87ad-7915154f4ac8\") "
Nov 03 16:58:24.881985 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.881963    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-openshift-service-ca" pod "f9cacf9e-deaf-41a7-87ad-7915154f4ac8" (UID: "f9cacf9e-deaf-41a7-87ad-7915154f4ac8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:58:24.882200 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.882180    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:24.883856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.883822    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-metrics-certs" pod "f9cacf9e-deaf-41a7-87ad-7915154f4ac8" (UID: "f9cacf9e-deaf-41a7-87ad-7915154f4ac8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:58:24.883980 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.883921    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-maas-api-tls" pod "f9cacf9e-deaf-41a7-87ad-7915154f4ac8" (UID: "f9cacf9e-deaf-41a7-87ad-7915154f4ac8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:58:24.884044 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.884003    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-kube-api-access-wzstk" pod "f9cacf9e-deaf-41a7-87ad-7915154f4ac8" (UID: "f9cacf9e-deaf-41a7-87ad-7915154f4ac8"). InnerVolumeSpecName "kube-api-access-wzstk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:58:24.983099 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.983006    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzstk\" (UniqueName: \"kubernetes.io/projected/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-kube-api-access-wzstk\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:24.983099 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.983039    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:24.983099 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:24.983049    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9cacf9e-deaf-41a7-87ad-7915154f4ac8-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:25.248403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:25.248310    5605 generic.go:356] "Generic (PLEG): container finished" podID="f9cacf9e-deaf-41a7-87ad-7915154f4ac8" containerID="c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413" exitCode=0
Nov 03 16:58:25.248403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:25.248362    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"
Nov 03 16:58:25.248634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:25.248400    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx" event={"ID":"f9cacf9e-deaf-41a7-87ad-7915154f4ac8","Type":"ContainerDied","Data":"c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413"}
Nov 03 16:58:25.248634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:25.248454    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx" event={"ID":"f9cacf9e-deaf-41a7-87ad-7915154f4ac8","Type":"ContainerDied","Data":"ad803734ccf9ca0f09848f150371f181563bf1745be9e01d161706bd961696ed"}
Nov 03 16:58:25.248634 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:25.248479    5605 scope.go:122] "RemoveContainer" containerID="c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413"
Nov 03 16:58:25.259109 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:25.259088    5605 scope.go:122] "RemoveContainer" containerID="c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413"
Nov 03 16:58:25.259371 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:58:25.259347    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413\": container with ID starting with c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413 not found: ID does not exist" containerID="c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413"
Nov 03 16:58:25.259441 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:25.259383    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413"} err="failed to get container status \"c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413\": rpc error: code = NotFound desc = could not find container \"c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413\": container with ID starting with c1dc5e591d652ce6fc8214a1ef0eb09105c2753a3346bb580a0ad9551c8c2413 not found: ID does not exist"
Nov 03 16:58:25.428121 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:25.428092    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"]
Nov 03 16:58:25.479129 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:25.479087    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4d9b-7698fb7854-4q7qx"]
Nov 03 16:58:26.309116 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:26.309081    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9cacf9e-deaf-41a7-87ad-7915154f4ac8" path="/var/lib/kubelet/pods/f9cacf9e-deaf-41a7-87ad-7915154f4ac8/volumes"
Nov 03 16:58:32.478397 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:32.478361    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x" podUID="29ac9424-33f5-48f0-83e2-022d1b036959" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 16:58:33.295024 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:33.294988    5605 generic.go:356] "Generic (PLEG): container finished" podID="29ac9424-33f5-48f0-83e2-022d1b036959" containerID="3e4b847b430db08147c4917c7e1e598a42c316ea688dc1f30f78a7b34854bbd3" exitCode=0
Nov 03 16:58:33.295196 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:33.295024    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x" event={"ID":"29ac9424-33f5-48f0-83e2-022d1b036959","Type":"ContainerDied","Data":"3e4b847b430db08147c4917c7e1e598a42c316ea688dc1f30f78a7b34854bbd3"}
Nov 03 16:58:33.372599 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:33.372575    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:58:33.393200 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:58:33.393159    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:58:33.486428 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:33.486390    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29ac9424-33f5-48f0-83e2-022d1b036959-kube-api-access-z2w2r\" (UniqueName: \"kubernetes.io/projected/29ac9424-33f5-48f0-83e2-022d1b036959-kube-api-access-z2w2r\") pod \"29ac9424-33f5-48f0-83e2-022d1b036959\" (UID: \"29ac9424-33f5-48f0-83e2-022d1b036959\") "
Nov 03 16:58:33.486851 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:33.486503    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29ac9424-33f5-48f0-83e2-022d1b036959-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29ac9424-33f5-48f0-83e2-022d1b036959-plugins-config-volume\") pod \"29ac9424-33f5-48f0-83e2-022d1b036959\" (UID: \"29ac9424-33f5-48f0-83e2-022d1b036959\") "
Nov 03 16:58:33.486927 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:33.486904    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29ac9424-33f5-48f0-83e2-022d1b036959-plugins-config-volume" pod "29ac9424-33f5-48f0-83e2-022d1b036959" (UID: "29ac9424-33f5-48f0-83e2-022d1b036959"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:58:33.489103 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:33.489073    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ac9424-33f5-48f0-83e2-022d1b036959-kube-api-access-z2w2r" pod "29ac9424-33f5-48f0-83e2-022d1b036959" (UID: "29ac9424-33f5-48f0-83e2-022d1b036959"). InnerVolumeSpecName "kube-api-access-z2w2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:58:33.587973 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:33.587935    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29ac9424-33f5-48f0-83e2-022d1b036959-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:33.587973 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:33.587967    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2w2r\" (UniqueName: \"kubernetes.io/projected/29ac9424-33f5-48f0-83e2-022d1b036959-kube-api-access-z2w2r\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:34.304333 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:34.304305    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"
Nov 03 16:58:34.308769 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:34.308717    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x" event={"ID":"29ac9424-33f5-48f0-83e2-022d1b036959","Type":"ContainerDied","Data":"a9fbbc30330182f3a219a581a0a537694e4aa40251567d206c9378e9dfc685bf"}
Nov 03 16:58:34.308901 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:34.308812    5605 scope.go:122] "RemoveContainer" containerID="3e4b847b430db08147c4917c7e1e598a42c316ea688dc1f30f78a7b34854bbd3"
Nov 03 16:58:34.475403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:34.475368    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"]
Nov 03 16:58:34.518322 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:34.518281    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4d9b-59b88977ff-bql7x"]
Nov 03 16:58:34.860247 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:34.860214    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"]
Nov 03 16:58:34.860471 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:34.860446    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy" containerID="cri-o://9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec" gracePeriod=30
Nov 03 16:58:36.308880 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:36.308844    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29ac9424-33f5-48f0-83e2-022d1b036959" path="/var/lib/kubelet/pods/29ac9424-33f5-48f0-83e2-022d1b036959/volumes"
Nov 03 16:58:39.053918 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:39.053882    5605 patch_prober.go:29] interesting pod/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 16:58:39.054327 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:39.053939    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 16:58:40.120110 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.120082    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:58:40.256537 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256446    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-socket\") pod \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") "
Nov 03 16:58:40.256697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256539    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-podinfo\") pod \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") "
Nov 03 16:58:40.256697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256593    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-credential-socket\") pod \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") "
Nov 03 16:58:40.256697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256615    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-data\") pod \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") "
Nov 03 16:58:40.256886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256780    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-certs\") pod \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") "
Nov 03 16:58:40.256886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256805    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-socket" pod "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" (UID: "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:58:40.256886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256840    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-envoy\") pod \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") "
Nov 03 16:58:40.256886 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256871    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istiod-ca-cert\") pod \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") "
Nov 03 16:58:40.257105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256929    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-data" pod "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" (UID: "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:58:40.257105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256961    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-kube-api-access-thg2t\" (UniqueName: \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-kube-api-access-thg2t\") pod \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") "
Nov 03 16:58:40.257105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256967    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-credential-socket" pod "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" (UID: "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:58:40.257105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.256999    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-token\" (UniqueName: \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-token\") pod \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\" (UID: \"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96\") "
Nov 03 16:58:40.257105 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.257085    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-certs" pod "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" (UID: "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:58:40.257364 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.257227    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istiod-ca-cert" pod "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" (UID: "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:58:40.257581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.257470    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:40.257581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.257502    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:40.257581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.257518    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:40.257581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.257533    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:40.257581 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.257547    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:40.259154 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.259130    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-podinfo" pod "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" (UID: "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 16:58:40.259277 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.259182    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-envoy" pod "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" (UID: "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:58:40.259277 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.259188    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-kube-api-access-thg2t" pod "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" (UID: "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96"). InnerVolumeSpecName "kube-api-access-thg2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:58:40.259614 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.259592    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-token" pod "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" (UID: "dcd0aeaa-25c4-49f1-8758-bbccf9f16b96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:58:40.336203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.336171    5605 generic.go:356] "Generic (PLEG): container finished" podID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerID="9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec" exitCode=0
Nov 03 16:58:40.336386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.336217    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" event={"ID":"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96","Type":"ContainerDied","Data":"9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec"}
Nov 03 16:58:40.336386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.336222    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"
Nov 03 16:58:40.336386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.336252    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv" event={"ID":"dcd0aeaa-25c4-49f1-8758-bbccf9f16b96","Type":"ContainerDied","Data":"75c02b88c0d224848728ffd1a51a935eeca469d6aed55374d281dae25c5b81dd"}
Nov 03 16:58:40.336386 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.336274    5605 scope.go:122] "RemoveContainer" containerID="9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec"
Nov 03 16:58:40.346428 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.346407    5605 scope.go:122] "RemoveContainer" containerID="af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83"
Nov 03 16:58:40.355623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.355593    5605 scope.go:122] "RemoveContainer" containerID="9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec"
Nov 03 16:58:40.355952 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:58:40.355930    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec\": container with ID starting with 9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec not found: ID does not exist" containerID="9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec"
Nov 03 16:58:40.356032 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.355961    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec"} err="failed to get container status \"9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec\": rpc error: code = NotFound desc = could not find container \"9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec\": container with ID starting with 9eb944cc88b58e1f77becd6760c056a98f5c6d496feaa9e24cde42a6a8e1a6ec not found: ID does not exist"
Nov 03 16:58:40.356032 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.355979    5605 scope.go:122] "RemoveContainer" containerID="af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83"
Nov 03 16:58:40.356225 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:58:40.356204    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83\": container with ID starting with af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83 not found: ID does not exist" containerID="af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83"
Nov 03 16:58:40.356266 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.356234    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83"} err="failed to get container status \"af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83\": rpc error: code = NotFound desc = could not find container \"af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83\": container with ID starting with af2fff26739bef64e635d690aadad978df0dfeba198b889d78d1fb2ef283eb83 not found: ID does not exist"
Nov 03 16:58:40.358951 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.358928    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:40.359022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.358952    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:40.359022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.358964    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thg2t\" (UniqueName: \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-kube-api-access-thg2t\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:40.359022 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.358973    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:58:40.510186 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.510097    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"]
Nov 03 16:58:40.516761 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:40.516716    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3b4d9b-openshift-default-59989c96c-fnfzv"]
Nov 03 16:58:42.309191 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:42.309157    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" path="/var/lib/kubelet/pods/dcd0aeaa-25c4-49f1-8758-bbccf9f16b96/volumes"
Nov 03 16:58:56.777143 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:58:56.777101    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"]
Nov 03 16:59:03.392863 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:59:03.392780    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:59:11.797709 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:11.797671    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c" podUID="2dc3e0cc-1f59-4681-906a-4cb13741e308" containerName="main" containerID="cri-o://eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7" gracePeriod=45
Nov 03 16:59:12.061040 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.060976    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:59:12.212999 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.212965    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-home\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-home\") pod \"2dc3e0cc-1f59-4681-906a-4cb13741e308\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") "
Nov 03 16:59:12.213203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.213045    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-tmp-dir\") pod \"2dc3e0cc-1f59-4681-906a-4cb13741e308\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") "
Nov 03 16:59:12.213203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.213090    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2dc3e0cc-1f59-4681-906a-4cb13741e308-tls-certs\" (UniqueName: \"kubernetes.io/secret/2dc3e0cc-1f59-4681-906a-4cb13741e308-tls-certs\") pod \"2dc3e0cc-1f59-4681-906a-4cb13741e308\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") "
Nov 03 16:59:12.213203 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.213157    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2dc3e0cc-1f59-4681-906a-4cb13741e308-kube-api-access-sncbt\" (UniqueName: \"kubernetes.io/projected/2dc3e0cc-1f59-4681-906a-4cb13741e308-kube-api-access-sncbt\") pod \"2dc3e0cc-1f59-4681-906a-4cb13741e308\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") "
Nov 03 16:59:12.213419 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.213246    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-home" pod "2dc3e0cc-1f59-4681-906a-4cb13741e308" (UID: "2dc3e0cc-1f59-4681-906a-4cb13741e308"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:59:12.213419 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.213273    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-dshm\") pod \"2dc3e0cc-1f59-4681-906a-4cb13741e308\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") "
Nov 03 16:59:12.213419 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.213332    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-tmp-dir" pod "2dc3e0cc-1f59-4681-906a-4cb13741e308" (UID: "2dc3e0cc-1f59-4681-906a-4cb13741e308"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:59:12.213419 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.213394    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-model-cache\") pod \"2dc3e0cc-1f59-4681-906a-4cb13741e308\" (UID: \"2dc3e0cc-1f59-4681-906a-4cb13741e308\") "
Nov 03 16:59:12.213690 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.213668    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-model-cache" pod "2dc3e0cc-1f59-4681-906a-4cb13741e308" (UID: "2dc3e0cc-1f59-4681-906a-4cb13741e308"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:59:12.214098 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.214079    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:12.214151 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.214104    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:12.214151 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.214119    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:12.215629 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.215603    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dc3e0cc-1f59-4681-906a-4cb13741e308-tls-certs" pod "2dc3e0cc-1f59-4681-906a-4cb13741e308" (UID: "2dc3e0cc-1f59-4681-906a-4cb13741e308"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:59:12.215815 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.215794    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dc3e0cc-1f59-4681-906a-4cb13741e308-kube-api-access-sncbt" pod "2dc3e0cc-1f59-4681-906a-4cb13741e308" (UID: "2dc3e0cc-1f59-4681-906a-4cb13741e308"). InnerVolumeSpecName "kube-api-access-sncbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:59:12.215932 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.215914    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-dshm" pod "2dc3e0cc-1f59-4681-906a-4cb13741e308" (UID: "2dc3e0cc-1f59-4681-906a-4cb13741e308"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 16:59:12.315403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.315335    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dc3e0cc-1f59-4681-906a-4cb13741e308-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:12.315403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.315367    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sncbt\" (UniqueName: \"kubernetes.io/projected/2dc3e0cc-1f59-4681-906a-4cb13741e308-kube-api-access-sncbt\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:12.315403 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.315381    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dc3e0cc-1f59-4681-906a-4cb13741e308-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:12.533666 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.533629    5605 generic.go:356] "Generic (PLEG): container finished" podID="2dc3e0cc-1f59-4681-906a-4cb13741e308" containerID="eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7" exitCode=0
Nov 03 16:59:12.533878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.533678    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"
Nov 03 16:59:12.533878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.533708    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c" event={"ID":"2dc3e0cc-1f59-4681-906a-4cb13741e308","Type":"ContainerDied","Data":"eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7"}
Nov 03 16:59:12.533878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.533772    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c" event={"ID":"2dc3e0cc-1f59-4681-906a-4cb13741e308","Type":"ContainerDied","Data":"21a6fb5c6823e1f6a888a547a4953d81db6e6a7094311a93d13e2c80f4e29d43"}
Nov 03 16:59:12.533878 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.533790    5605 scope.go:122] "RemoveContainer" containerID="eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7"
Nov 03 16:59:12.543474 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.543449    5605 scope.go:122] "RemoveContainer" containerID="eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7"
Nov 03 16:59:12.543772 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:59:12.543753    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7\": container with ID starting with eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7 not found: ID does not exist" containerID="eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7"
Nov 03 16:59:12.543840 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.543779    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7"} err="failed to get container status \"eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7\": rpc error: code = NotFound desc = could not find container \"eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7\": container with ID starting with eb08725c59dc4d78a58fc5c47f47b2f7743cc5c9ea3d9dd97970ab44841fcbe7 not found: ID does not exist"
Nov 03 16:59:12.707911 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.707868    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"]
Nov 03 16:59:12.713419 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:12.713395    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c"]
Nov 03 16:59:13.037766 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:13.037641    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-7e55c3/test-model-7e55c3-kserve-768966df8c-b2w5c" podUID="2dc3e0cc-1f59-4681-906a-4cb13741e308" containerName="main" probeResult="failure" output="Get \"https://10.133.0.142:8000/ready\": context deadline exceeded"
Nov 03 16:59:14.308004 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:14.307968    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2dc3e0cc-1f59-4681-906a-4cb13741e308" path="/var/lib/kubelet/pods/2dc3e0cc-1f59-4681-906a-4cb13741e308/volumes"
Nov 03 16:59:26.355704 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:26.355668    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"]
Nov 03 16:59:26.462278 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:26.462242    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"]
Nov 03 16:59:31.356295 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.356235    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk" podUID="8c1cccb3-fd69-413b-9974-b1d1d9c27899" containerName="payload-processing" containerID="cri-o://f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f" gracePeriod=25
Nov 03 16:59:31.462832 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.462790    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t" podUID="f7cb22e7-2865-4211-a1a4-12eba70bd7b2" containerName="payload-pre-processing" containerID="cri-o://64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204" gracePeriod=25
Nov 03 16:59:31.613674 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.613619    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:59:31.632208 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.632178    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c1cccb3-fd69-413b-9974-b1d1d9c27899-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c1cccb3-fd69-413b-9974-b1d1d9c27899-plugins-config-volume\") pod \"8c1cccb3-fd69-413b-9974-b1d1d9c27899\" (UID: \"8c1cccb3-fd69-413b-9974-b1d1d9c27899\") "
Nov 03 16:59:31.632396 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.632382    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c1cccb3-fd69-413b-9974-b1d1d9c27899-kube-api-access-g8kjj\" (UniqueName: \"kubernetes.io/projected/8c1cccb3-fd69-413b-9974-b1d1d9c27899-kube-api-access-g8kjj\") pod \"8c1cccb3-fd69-413b-9974-b1d1d9c27899\" (UID: \"8c1cccb3-fd69-413b-9974-b1d1d9c27899\") "
Nov 03 16:59:31.632514 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.632492    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c1cccb3-fd69-413b-9974-b1d1d9c27899-plugins-config-volume" pod "8c1cccb3-fd69-413b-9974-b1d1d9c27899" (UID: "8c1cccb3-fd69-413b-9974-b1d1d9c27899"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:59:31.632636 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.632621    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c1cccb3-fd69-413b-9974-b1d1d9c27899-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:31.634586 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.634563    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c1cccb3-fd69-413b-9974-b1d1d9c27899-kube-api-access-g8kjj" pod "8c1cccb3-fd69-413b-9974-b1d1d9c27899" (UID: "8c1cccb3-fd69-413b-9974-b1d1d9c27899"). InnerVolumeSpecName "kube-api-access-g8kjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:59:31.638392 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.638364    5605 generic.go:356] "Generic (PLEG): container finished" podID="8c1cccb3-fd69-413b-9974-b1d1d9c27899" containerID="f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f" exitCode=0
Nov 03 16:59:31.638489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.638401    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk" event={"ID":"8c1cccb3-fd69-413b-9974-b1d1d9c27899","Type":"ContainerDied","Data":"f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f"}
Nov 03 16:59:31.638489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.638429    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"
Nov 03 16:59:31.638489 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.638480    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk" event={"ID":"8c1cccb3-fd69-413b-9974-b1d1d9c27899","Type":"ContainerDied","Data":"6e6bb2702c7c70e0855bcd9609fe081db5d73340a3d3c8ea2d3aa87e7230ff35"}
Nov 03 16:59:31.638594 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.638505    5605 scope.go:122] "RemoveContainer" containerID="f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f"
Nov 03 16:59:31.650291 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.650275    5605 scope.go:122] "RemoveContainer" containerID="f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f"
Nov 03 16:59:31.650557 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:59:31.650539    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f\": container with ID starting with f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f not found: ID does not exist" containerID="f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f"
Nov 03 16:59:31.650609 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.650564    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f"} err="failed to get container status \"f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f\": rpc error: code = NotFound desc = could not find container \"f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f\": container with ID starting with f1285b20601c9bab8c04683c07b04fb0d75529a690fa2b03070cc9d2aa7b516f not found: ID does not exist"
Nov 03 16:59:31.733406 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.733367    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8kjj\" (UniqueName: \"kubernetes.io/projected/8c1cccb3-fd69-413b-9974-b1d1d9c27899-kube-api-access-g8kjj\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:31.851712 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.851677    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"]
Nov 03 16:59:31.859856 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:31.859831    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7e55c3-595cfc4f6f-jv8pk"]
Nov 03 16:59:32.309996 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:32.309962    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c1cccb3-fd69-413b-9974-b1d1d9c27899" path="/var/lib/kubelet/pods/8c1cccb3-fd69-413b-9974-b1d1d9c27899/volumes"
Nov 03 16:59:33.392280 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:59:33.392240    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 16:59:37.352049 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:37.352011    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t" podUID="f7cb22e7-2865-4211-a1a4-12eba70bd7b2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 16:59:47.351653 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:47.351612    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t" podUID="f7cb22e7-2865-4211-a1a4-12eba70bd7b2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 16:59:48.415937 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.415898    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"]
Nov 03 16:59:48.416368 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.416105    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn" podUID="a4250677-5a92-4fe9-88d5-66670ff0dfff" containerName="maas-api" containerID="cri-o://e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2" gracePeriod=30
Nov 03 16:59:48.685385 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.685356    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:59:48.729053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.729022    5605 generic.go:356] "Generic (PLEG): container finished" podID="a4250677-5a92-4fe9-88d5-66670ff0dfff" containerID="e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2" exitCode=0
Nov 03 16:59:48.729232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.729070    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"
Nov 03 16:59:48.729232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.729110    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn" event={"ID":"a4250677-5a92-4fe9-88d5-66670ff0dfff","Type":"ContainerDied","Data":"e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2"}
Nov 03 16:59:48.729232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.729199    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn" event={"ID":"a4250677-5a92-4fe9-88d5-66670ff0dfff","Type":"ContainerDied","Data":"78429fa14ef3d73cdcf3cdd7de6ce7543c70d71a3d9c5c98be89393787b6ec14"}
Nov 03 16:59:48.729232 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.729215    5605 scope.go:122] "RemoveContainer" containerID="e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2"
Nov 03 16:59:48.738389 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.738371    5605 scope.go:122] "RemoveContainer" containerID="e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2"
Nov 03 16:59:48.738684 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:59:48.738664    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2\": container with ID starting with e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2 not found: ID does not exist" containerID="e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2"
Nov 03 16:59:48.738803 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.738695    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2"} err="failed to get container status \"e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2\": rpc error: code = NotFound desc = could not find container \"e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2\": container with ID starting with e55d1abaad8180e4404537cb2a5079261afe44cc9f9e657d0be0cc5179ba82b2 not found: ID does not exist"
Nov 03 16:59:48.828231 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.828196    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4250677-5a92-4fe9-88d5-66670ff0dfff-kube-api-access-cfg6j\" (UniqueName: \"kubernetes.io/projected/a4250677-5a92-4fe9-88d5-66670ff0dfff-kube-api-access-cfg6j\") pod \"a4250677-5a92-4fe9-88d5-66670ff0dfff\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") "
Nov 03 16:59:48.828425 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.828370    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-metrics-certs\") pod \"a4250677-5a92-4fe9-88d5-66670ff0dfff\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") "
Nov 03 16:59:48.828495 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.828469    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4250677-5a92-4fe9-88d5-66670ff0dfff-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4250677-5a92-4fe9-88d5-66670ff0dfff-openshift-service-ca\") pod \"a4250677-5a92-4fe9-88d5-66670ff0dfff\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") "
Nov 03 16:59:48.828549 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.828509    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-maas-api-tls\") pod \"a4250677-5a92-4fe9-88d5-66670ff0dfff\" (UID: \"a4250677-5a92-4fe9-88d5-66670ff0dfff\") "
Nov 03 16:59:48.829444 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.829409    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4250677-5a92-4fe9-88d5-66670ff0dfff-openshift-service-ca" pod "a4250677-5a92-4fe9-88d5-66670ff0dfff" (UID: "a4250677-5a92-4fe9-88d5-66670ff0dfff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:59:48.832801 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.832768    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-maas-api-tls" pod "a4250677-5a92-4fe9-88d5-66670ff0dfff" (UID: "a4250677-5a92-4fe9-88d5-66670ff0dfff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:59:48.834432 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.834405    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4250677-5a92-4fe9-88d5-66670ff0dfff-kube-api-access-cfg6j" pod "a4250677-5a92-4fe9-88d5-66670ff0dfff" (UID: "a4250677-5a92-4fe9-88d5-66670ff0dfff"). InnerVolumeSpecName "kube-api-access-cfg6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:59:48.834629 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.834607    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-metrics-certs" pod "a4250677-5a92-4fe9-88d5-66670ff0dfff" (UID: "a4250677-5a92-4fe9-88d5-66670ff0dfff"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 16:59:48.932053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.931468    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfg6j\" (UniqueName: \"kubernetes.io/projected/a4250677-5a92-4fe9-88d5-66670ff0dfff-kube-api-access-cfg6j\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:48.932053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.931511    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:48.932053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.931529    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4250677-5a92-4fe9-88d5-66670ff0dfff-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:48.932053 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:48.931543    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4250677-5a92-4fe9-88d5-66670ff0dfff-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:49.157871 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:49.157840    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"]
Nov 03 16:59:49.164893 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:49.164867    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn"]
Nov 03 16:59:50.309314 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:50.309280    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4250677-5a92-4fe9-88d5-66670ff0dfff" path="/var/lib/kubelet/pods/a4250677-5a92-4fe9-88d5-66670ff0dfff/volumes"
Nov 03 16:59:51.648921 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.648859    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7e55c3-5db4547fcb-4nvwn" podUID="a4250677-5a92-4fe9-88d5-66670ff0dfff" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.139:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 16:59:51.729657 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.729586    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:59:51.746417 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.746384    5605 generic.go:356] "Generic (PLEG): container finished" podID="f7cb22e7-2865-4211-a1a4-12eba70bd7b2" containerID="64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204" exitCode=0
Nov 03 16:59:51.746551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.746438    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"
Nov 03 16:59:51.746551 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.746468    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t" event={"ID":"f7cb22e7-2865-4211-a1a4-12eba70bd7b2","Type":"ContainerDied","Data":"64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204"}
Nov 03 16:59:51.746623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.746562    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t" event={"ID":"f7cb22e7-2865-4211-a1a4-12eba70bd7b2","Type":"ContainerDied","Data":"a7fb6a7ffa845b4c9a516ab421104c04a2740c494259a64d22026e48083facd8"}
Nov 03 16:59:51.746623 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.746593    5605 scope.go:122] "RemoveContainer" containerID="64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204"
Nov 03 16:59:51.756697 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.756675    5605 scope.go:122] "RemoveContainer" containerID="64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204"
Nov 03 16:59:51.756986 ip-10-0-142-22 kubenswrapper[5605]: E1103 16:59:51.756962    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204\": container with ID starting with 64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204 not found: ID does not exist" containerID="64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204"
Nov 03 16:59:51.757055 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.756996    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204"} err="failed to get container status \"64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204\": rpc error: code = NotFound desc = could not find container \"64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204\": container with ID starting with 64c36eee37aafdf11c3fe6abbe4e16d9156f540ae5450a031d8f4776f657a204 not found: ID does not exist"
Nov 03 16:59:51.861425 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.861397    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-kube-api-access-v2lc8\" (UniqueName: \"kubernetes.io/projected/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-kube-api-access-v2lc8\") pod \"f7cb22e7-2865-4211-a1a4-12eba70bd7b2\" (UID: \"f7cb22e7-2865-4211-a1a4-12eba70bd7b2\") "
Nov 03 16:59:51.861658 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.861621    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-plugins-config-volume\") pod \"f7cb22e7-2865-4211-a1a4-12eba70bd7b2\" (UID: \"f7cb22e7-2865-4211-a1a4-12eba70bd7b2\") "
Nov 03 16:59:51.862041 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.862009    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-plugins-config-volume" pod "f7cb22e7-2865-4211-a1a4-12eba70bd7b2" (UID: "f7cb22e7-2865-4211-a1a4-12eba70bd7b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 16:59:51.864182 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.864154    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-kube-api-access-v2lc8" pod "f7cb22e7-2865-4211-a1a4-12eba70bd7b2" (UID: "f7cb22e7-2865-4211-a1a4-12eba70bd7b2"). InnerVolumeSpecName "kube-api-access-v2lc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 16:59:51.963475 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.963435    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2lc8\" (UniqueName: \"kubernetes.io/projected/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-kube-api-access-v2lc8\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:51.963475 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:51.963469    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7cb22e7-2865-4211-a1a4-12eba70bd7b2-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 16:59:52.172502 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:52.172468    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"]
Nov 03 16:59:52.178580 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:52.178553    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7e55c3-5c77dbbdc9-6jk8t"]
Nov 03 16:59:52.307894 ip-10-0-142-22 kubenswrapper[5605]: I1103 16:59:52.307820    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7cb22e7-2865-4211-a1a4-12eba70bd7b2" path="/var/lib/kubelet/pods/f7cb22e7-2865-4211-a1a4-12eba70bd7b2/volumes"
Nov 03 17:00:00.186907 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.186877    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"]
Nov 03 17:00:00.187357 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187343    5605 cpu_manager.go:414] "containerMap: removing container" podUID="8c1cccb3-fd69-413b-9974-b1d1d9c27899" containerName="payload-processing"
Nov 03 17:00:00.187402 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187359    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c1cccb3-fd69-413b-9974-b1d1d9c27899" containerName="payload-processing"
Nov 03 17:00:00.187402 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187372    5605 cpu_manager.go:414] "containerMap: removing container" podUID="a4250677-5a92-4fe9-88d5-66670ff0dfff" containerName="maas-api"
Nov 03 17:00:00.187402 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187378    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4250677-5a92-4fe9-88d5-66670ff0dfff" containerName="maas-api"
Nov 03 17:00:00.187402 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187390    5605 cpu_manager.go:414] "containerMap: removing container" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy"
Nov 03 17:00:00.187402 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187395    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy"
Nov 03 17:00:00.187402 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187401    5605 cpu_manager.go:414] "containerMap: removing container" podUID="2dc3e0cc-1f59-4681-906a-4cb13741e308" containerName="main"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187406    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2dc3e0cc-1f59-4681-906a-4cb13741e308" containerName="main"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187419    5605 cpu_manager.go:414] "containerMap: removing container" podUID="29ac9424-33f5-48f0-83e2-022d1b036959" containerName="payload-pre-processing"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187430    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29ac9424-33f5-48f0-83e2-022d1b036959" containerName="payload-pre-processing"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187438    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187442    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187451    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f9cacf9e-deaf-41a7-87ad-7915154f4ac8" containerName="maas-api"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187455    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9cacf9e-deaf-41a7-87ad-7915154f4ac8" containerName="maas-api"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187462    5605 cpu_manager.go:414] "containerMap: removing container" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187467    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187473    5605 cpu_manager.go:414] "containerMap: removing container" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187477    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187488    5605 cpu_manager.go:414] "containerMap: removing container" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187495    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187505    5605 cpu_manager.go:414] "containerMap: removing container" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187510    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187517    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f5c1f5f4-78bb-4305-9213-1660fcc23cbf" containerName="payload-processing"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187521    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5c1f5f4-78bb-4305-9213-1660fcc23cbf" containerName="payload-processing"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187532    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f7cb22e7-2865-4211-a1a4-12eba70bd7b2" containerName="payload-pre-processing"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187536    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7cb22e7-2865-4211-a1a4-12eba70bd7b2" containerName="payload-pre-processing"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187549    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187553    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187559    5605 cpu_manager.go:414] "containerMap: removing container" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy"
Nov 03 17:00:00.187591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187562    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187630    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187637    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae2b9d64-981e-419f-84be-3beae332c9d1" containerName="istio-proxy"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187644    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187655    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c1cccb3-fd69-413b-9974-b1d1d9c27899" containerName="payload-processing"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187663    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4250677-5a92-4fe9-88d5-66670ff0dfff" containerName="maas-api"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187671    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187675    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="2dc3e0cc-1f59-4681-906a-4cb13741e308" containerName="main"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187681    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="29ac9424-33f5-48f0-83e2-022d1b036959" containerName="payload-pre-processing"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187686    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca21abb4-1609-4b5e-81b8-72cdc2481417" containerName="istio-proxy"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187691    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9cacf9e-deaf-41a7-87ad-7915154f4ac8" containerName="maas-api"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187696    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187701    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9c549fd-9a32-4a08-a3ce-bf698639f434" containerName="istio-proxy"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187707    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b444a3c-d720-476d-81b9-b4e15407c2e3" containerName="istio-proxy"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187712    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcd0aeaa-25c4-49f1-8758-bbccf9f16b96" containerName="istio-proxy"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187717    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5c1f5f4-78bb-4305-9213-1660fcc23cbf" containerName="payload-processing"
Nov 03 17:00:00.188279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.187742    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7cb22e7-2865-4211-a1a4-12eba70bd7b2" containerName="payload-pre-processing"
Nov 03 17:00:00.192675 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.192524    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"
Nov 03 17:00:00.212217 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.212190    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"]
Nov 03 17:00:00.213366 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.213341    5605 cpu_manager.go:414] "containerMap: removing container" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy"
Nov 03 17:00:00.213366 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.213366    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfd53f66-eba2-4ad7-a908-a42c9b6914d9" containerName="istio-proxy"
Nov 03 17:00:00.217004 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.216980    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"
Nov 03 17:00:00.219081 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.219047    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"]
Nov 03 17:00:00.222883 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.222861    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"
Nov 03 17:00:00.276594 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.276560    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"]
Nov 03 17:00:00.291281 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.291254    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fnmxp\""
Nov 03 17:00:00.311276 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.311245    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"]
Nov 03 17:00:00.316989 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.316961    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"]
Nov 03 17:00:00.358575 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.358537    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bea05f-9f14-40e0-898a-b957b279858f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw\" (UID: \"97bea05f-9f14-40e0-898a-b957b279858f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"
Nov 03 17:00:00.358771 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.358623    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6glc\" (UniqueName: \"kubernetes.io/projected/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-kube-api-access-m6glc\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc\" (UID: \"30f6c764-7a1b-48e0-9b28-ddeb3b151f06\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"
Nov 03 17:00:00.358771 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.358663    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b378df9b-dd3b-40c3-8886-1b67696431b3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc\" (UID: \"b378df9b-dd3b-40c3-8886-1b67696431b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"
Nov 03 17:00:00.358866 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.358779    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2sbj\" (UniqueName: \"kubernetes.io/projected/b378df9b-dd3b-40c3-8886-1b67696431b3-kube-api-access-r2sbj\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc\" (UID: \"b378df9b-dd3b-40c3-8886-1b67696431b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"
Nov 03 17:00:00.358936 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.358917    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plfs9\" (UniqueName: \"kubernetes.io/projected/97bea05f-9f14-40e0-898a-b957b279858f-kube-api-access-plfs9\") pod \"maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw\" (UID: \"97bea05f-9f14-40e0-898a-b957b279858f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"
Nov 03 17:00:00.358988 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.358967    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc\" (UID: \"30f6c764-7a1b-48e0-9b28-ddeb3b151f06\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"
Nov 03 17:00:00.459574 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.459484    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bea05f-9f14-40e0-898a-b957b279858f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw\" (UID: \"97bea05f-9f14-40e0-898a-b957b279858f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"
Nov 03 17:00:00.459574 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.459530    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6glc\" (UniqueName: \"kubernetes.io/projected/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-kube-api-access-m6glc\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc\" (UID: \"30f6c764-7a1b-48e0-9b28-ddeb3b151f06\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"
Nov 03 17:00:00.459574 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.459566    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b378df9b-dd3b-40c3-8886-1b67696431b3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc\" (UID: \"b378df9b-dd3b-40c3-8886-1b67696431b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"
Nov 03 17:00:00.459897 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.459595    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2sbj\" (UniqueName: \"kubernetes.io/projected/b378df9b-dd3b-40c3-8886-1b67696431b3-kube-api-access-r2sbj\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc\" (UID: \"b378df9b-dd3b-40c3-8886-1b67696431b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"
Nov 03 17:00:00.459897 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.459639    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plfs9\" (UniqueName: \"kubernetes.io/projected/97bea05f-9f14-40e0-898a-b957b279858f-kube-api-access-plfs9\") pod \"maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw\" (UID: \"97bea05f-9f14-40e0-898a-b957b279858f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"
Nov 03 17:00:00.459897 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.459665    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc\" (UID: \"30f6c764-7a1b-48e0-9b28-ddeb3b151f06\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"
Nov 03 17:00:00.460259 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.460238    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bea05f-9f14-40e0-898a-b957b279858f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw\" (UID: \"97bea05f-9f14-40e0-898a-b957b279858f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"
Nov 03 17:00:00.460336 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.460317    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b378df9b-dd3b-40c3-8886-1b67696431b3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc\" (UID: \"b378df9b-dd3b-40c3-8886-1b67696431b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"
Nov 03 17:00:00.460439 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.460420    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc\" (UID: \"30f6c764-7a1b-48e0-9b28-ddeb3b151f06\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"
Nov 03 17:00:00.520276 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.520240    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6glc\" (UniqueName: \"kubernetes.io/projected/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-kube-api-access-m6glc\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc\" (UID: \"30f6c764-7a1b-48e0-9b28-ddeb3b151f06\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"
Nov 03 17:00:00.520659 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.520640    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plfs9\" (UniqueName: \"kubernetes.io/projected/97bea05f-9f14-40e0-898a-b957b279858f-kube-api-access-plfs9\") pod \"maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw\" (UID: \"97bea05f-9f14-40e0-898a-b957b279858f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"
Nov 03 17:00:00.520750 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.520706    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2sbj\" (UniqueName: \"kubernetes.io/projected/b378df9b-dd3b-40c3-8886-1b67696431b3-kube-api-access-r2sbj\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc\" (UID: \"b378df9b-dd3b-40c3-8886-1b67696431b3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"
Nov 03 17:00:00.532577 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.532545    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"
Nov 03 17:00:00.540834 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.540814    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"
Nov 03 17:00:00.814847 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.814777    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"
Nov 03 17:00:00.930673 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.930604    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"]
Nov 03 17:00:00.933331 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:00:00.933301    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97bea05f_9f14_40e0_898a_b957b279858f.slice/crio-f8add702ac417f1866bd61363c60c01553c86c4659cf1ed571f16ca7f652012b WatchSource:0}: Error finding container f8add702ac417f1866bd61363c60c01553c86c4659cf1ed571f16ca7f652012b: Status 404 returned error can't find the container with id f8add702ac417f1866bd61363c60c01553c86c4659cf1ed571f16ca7f652012b
Nov 03 17:00:00.982989 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:00.982961    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"]
Nov 03 17:00:00.984992 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:00:00.984964    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30f6c764_7a1b_48e0_9b28_ddeb3b151f06.slice/crio-4b48bb03325b7cd67d8bb8095071cbf0919273cb5c7aa00c28f02dc5d8b0d650 WatchSource:0}: Error finding container 4b48bb03325b7cd67d8bb8095071cbf0919273cb5c7aa00c28f02dc5d8b0d650: Status 404 returned error can't find the container with id 4b48bb03325b7cd67d8bb8095071cbf0919273cb5c7aa00c28f02dc5d8b0d650
Nov 03 17:00:01.195292 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.195265    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"]
Nov 03 17:00:01.196775 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:00:01.196750    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb378df9b_dd3b_40c3_8886_1b67696431b3.slice/crio-166a820fffe79a9f1e5a0b4b22d1566a6f5a6fb42091ef7ce017cdddf767848a WatchSource:0}: Error finding container 166a820fffe79a9f1e5a0b4b22d1566a6f5a6fb42091ef7ce017cdddf767848a: Status 404 returned error can't find the container with id 166a820fffe79a9f1e5a0b4b22d1566a6f5a6fb42091ef7ce017cdddf767848a
Nov 03 17:00:01.799948 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.799917    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" event={"ID":"b378df9b-dd3b-40c3-8886-1b67696431b3","Type":"ContainerStarted","Data":"6f776e1c3581cdd8f0db9b3c1dce87fa11021284b01ae5f9b78e21a64f6fd52d"}
Nov 03 17:00:01.800048 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.799982    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" event={"ID":"b378df9b-dd3b-40c3-8886-1b67696431b3","Type":"ContainerStarted","Data":"166a820fffe79a9f1e5a0b4b22d1566a6f5a6fb42091ef7ce017cdddf767848a"}
Nov 03 17:00:01.801455 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.801430    5605 generic.go:356] "Generic (PLEG): container finished" podID="97bea05f-9f14-40e0-898a-b957b279858f" containerID="eb01039339fc8e698f3856672f704c801e529a99fc425f180f348c9bf81b3485" exitCode=60
Nov 03 17:00:01.801551 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.801507    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw" event={"ID":"97bea05f-9f14-40e0-898a-b957b279858f","Type":"ContainerDied","Data":"eb01039339fc8e698f3856672f704c801e529a99fc425f180f348c9bf81b3485"}
Nov 03 17:00:01.801596 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.801557    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw" event={"ID":"97bea05f-9f14-40e0-898a-b957b279858f","Type":"ContainerStarted","Data":"f8add702ac417f1866bd61363c60c01553c86c4659cf1ed571f16ca7f652012b"}
Nov 03 17:00:01.801765 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.801742    5605 scope.go:122] "RemoveContainer" containerID="eb01039339fc8e698f3856672f704c801e529a99fc425f180f348c9bf81b3485"
Nov 03 17:00:01.803271 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.803171    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" event={"ID":"30f6c764-7a1b-48e0-9b28-ddeb3b151f06","Type":"ContainerStarted","Data":"a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420"}
Nov 03 17:00:01.803271 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.803217    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" event={"ID":"30f6c764-7a1b-48e0-9b28-ddeb3b151f06","Type":"ContainerStarted","Data":"4b48bb03325b7cd67d8bb8095071cbf0919273cb5c7aa00c28f02dc5d8b0d650"}
Nov 03 17:00:01.803463 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.803447    5605 scope.go:122] "RemoveContainer" containerID="a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420"
Nov 03 17:00:01.923707 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:01.923646    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" podStartSLOduration=1.9236330879999999 podStartE2EDuration="1.923633088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:00:01.921566121 +0000 UTC m=+2887.746017393" watchObservedRunningTime="2026-11-03 17:00:01.923633088 +0000 UTC m=+2887.748084359"
Nov 03 17:00:02.074479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.074440    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"]
Nov 03 17:00:02.176564 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.176515    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"]
Nov 03 17:00:02.810453 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.810414    5605 generic.go:356] "Generic (PLEG): container finished" podID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerID="6f776e1c3581cdd8f0db9b3c1dce87fa11021284b01ae5f9b78e21a64f6fd52d" exitCode=60
Nov 03 17:00:02.810932 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.810505    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" event={"ID":"b378df9b-dd3b-40c3-8886-1b67696431b3","Type":"ContainerDied","Data":"6f776e1c3581cdd8f0db9b3c1dce87fa11021284b01ae5f9b78e21a64f6fd52d"}
Nov 03 17:00:02.810932 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.810797    5605 scope.go:122] "RemoveContainer" containerID="6f776e1c3581cdd8f0db9b3c1dce87fa11021284b01ae5f9b78e21a64f6fd52d"
Nov 03 17:00:02.812595 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.812567    5605 generic.go:356] "Generic (PLEG): container finished" podID="97bea05f-9f14-40e0-898a-b957b279858f" containerID="eb724a38bcccca93ca6f219e1b247ba0aa900b05545316a5c9ace46b9ad04b10" exitCode=60
Nov 03 17:00:02.812752 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.812636    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw" event={"ID":"97bea05f-9f14-40e0-898a-b957b279858f","Type":"ContainerDied","Data":"eb724a38bcccca93ca6f219e1b247ba0aa900b05545316a5c9ace46b9ad04b10"}
Nov 03 17:00:02.812752 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.812686    5605 scope.go:122] "RemoveContainer" containerID="eb01039339fc8e698f3856672f704c801e529a99fc425f180f348c9bf81b3485"
Nov 03 17:00:02.812952 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.812936    5605 scope.go:122] "RemoveContainer" containerID="eb724a38bcccca93ca6f219e1b247ba0aa900b05545316a5c9ace46b9ad04b10"
Nov 03 17:00:02.813679 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:02.813657    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw_odh-ai-gateway-infra(97bea05f-9f14-40e0-898a-b957b279858f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw" podUID="97bea05f-9f14-40e0-898a-b957b279858f"
Nov 03 17:00:02.814684 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.814664    5605 generic.go:356] "Generic (PLEG): container finished" podID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerID="a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420" exitCode=60
Nov 03 17:00:02.814684 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.814683    5605 generic.go:356] "Generic (PLEG): container finished" podID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerID="3d1715bee3da4c9e11f5a870d81d8c669f921ad3281b60c6067eb88ede7e50d3" exitCode=60
Nov 03 17:00:02.814832 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.814692    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" event={"ID":"30f6c764-7a1b-48e0-9b28-ddeb3b151f06","Type":"ContainerDied","Data":"a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420"}
Nov 03 17:00:02.814832 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.814770    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" event={"ID":"30f6c764-7a1b-48e0-9b28-ddeb3b151f06","Type":"ContainerDied","Data":"3d1715bee3da4c9e11f5a870d81d8c669f921ad3281b60c6067eb88ede7e50d3"}
Nov 03 17:00:02.814994 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.814981    5605 scope.go:122] "RemoveContainer" containerID="3d1715bee3da4c9e11f5a870d81d8c669f921ad3281b60c6067eb88ede7e50d3"
Nov 03 17:00:02.815196 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:02.815179    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc_odh-ai-gateway-infra(30f6c764-7a1b-48e0-9b28-ddeb3b151f06)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06"
Nov 03 17:00:02.825099 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.825079    5605 scope.go:122] "RemoveContainer" containerID="a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420"
Nov 03 17:00:02.840553 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.840534    5605 scope.go:122] "RemoveContainer" containerID="a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420"
Nov 03 17:00:02.840945 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:02.840922    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420\": container with ID starting with a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420 not found: ID does not exist" containerID="a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420"
Nov 03 17:00:02.841020 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:02.840955    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420"} err="failed to get container status \"a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420\": rpc error: code = NotFound desc = could not find container \"a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420\": container with ID starting with a56d1f49818b0bebb2927ded881a26b9bf7c312250c8a550acd314dc7e61c420 not found: ID does not exist"
Nov 03 17:00:03.394185 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:03.394147    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:00:03.822152 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:03.822068    5605 scope.go:122] "RemoveContainer" containerID="3d1715bee3da4c9e11f5a870d81d8c669f921ad3281b60c6067eb88ede7e50d3"
Nov 03 17:00:03.822598 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:03.822378    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc_odh-ai-gateway-infra(30f6c764-7a1b-48e0-9b28-ddeb3b151f06)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06"
Nov 03 17:00:03.823814 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:03.823784    5605 generic.go:356] "Generic (PLEG): container finished" podID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerID="d2b14140f81438c44cdc186ea785b78914780f7f3b81beabdf43d17aab4bead8" exitCode=60
Nov 03 17:00:03.823929 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:03.823869    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" event={"ID":"b378df9b-dd3b-40c3-8886-1b67696431b3","Type":"ContainerDied","Data":"d2b14140f81438c44cdc186ea785b78914780f7f3b81beabdf43d17aab4bead8"}
Nov 03 17:00:03.823994 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:03.823958    5605 scope.go:122] "RemoveContainer" containerID="6f776e1c3581cdd8f0db9b3c1dce87fa11021284b01ae5f9b78e21a64f6fd52d"
Nov 03 17:00:03.824217 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:03.824192    5605 scope.go:122] "RemoveContainer" containerID="d2b14140f81438c44cdc186ea785b78914780f7f3b81beabdf43d17aab4bead8"
Nov 03 17:00:03.824502 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:03.824482    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc_odh-ai-gateway-infra(b378df9b-dd3b-40c3-8886-1b67696431b3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3"
Nov 03 17:00:03.825707 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:03.825689    5605 scope.go:122] "RemoveContainer" containerID="eb724a38bcccca93ca6f219e1b247ba0aa900b05545316a5c9ace46b9ad04b10"
Nov 03 17:00:03.825915 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:03.825895    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw_odh-ai-gateway-infra(97bea05f-9f14-40e0-898a-b957b279858f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw" podUID="97bea05f-9f14-40e0-898a-b957b279858f"
Nov 03 17:00:04.831992 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:04.831960    5605 scope.go:122] "RemoveContainer" containerID="d2b14140f81438c44cdc186ea785b78914780f7f3b81beabdf43d17aab4bead8"
Nov 03 17:00:04.832384 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:04.832200    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc_odh-ai-gateway-infra(b378df9b-dd3b-40c3-8886-1b67696431b3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3"
Nov 03 17:00:07.075023 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.074975    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj" podUID="9744816f-937a-43ab-b49d-8cfc4c4522dc" containerName="payload-processing" containerID="cri-o://6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834" gracePeriod=25
Nov 03 17:00:07.176980 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.176939    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw" podUID="0233caa0-0726-4bd8-a510-fbc1799b3eab" containerName="payload-pre-processing" containerID="cri-o://5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688" gracePeriod=25
Nov 03 17:00:07.341285 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.341258    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 17:00:07.438859 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.438835    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 17:00:07.450425 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.450402    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9744816f-937a-43ab-b49d-8cfc4c4522dc-kube-api-access-c9vd8\" (UniqueName: \"kubernetes.io/projected/9744816f-937a-43ab-b49d-8cfc4c4522dc-kube-api-access-c9vd8\") pod \"9744816f-937a-43ab-b49d-8cfc4c4522dc\" (UID: \"9744816f-937a-43ab-b49d-8cfc4c4522dc\") "
Nov 03 17:00:07.450543 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.450433    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9744816f-937a-43ab-b49d-8cfc4c4522dc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9744816f-937a-43ab-b49d-8cfc4c4522dc-plugins-config-volume\") pod \"9744816f-937a-43ab-b49d-8cfc4c4522dc\" (UID: \"9744816f-937a-43ab-b49d-8cfc4c4522dc\") "
Nov 03 17:00:07.450823 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.450798    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9744816f-937a-43ab-b49d-8cfc4c4522dc-plugins-config-volume" pod "9744816f-937a-43ab-b49d-8cfc4c4522dc" (UID: "9744816f-937a-43ab-b49d-8cfc4c4522dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:00:07.452640 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.452614    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9744816f-937a-43ab-b49d-8cfc4c4522dc-kube-api-access-c9vd8" pod "9744816f-937a-43ab-b49d-8cfc4c4522dc" (UID: "9744816f-937a-43ab-b49d-8cfc4c4522dc"). InnerVolumeSpecName "kube-api-access-c9vd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:00:07.551770 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.551716    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0233caa0-0726-4bd8-a510-fbc1799b3eab-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0233caa0-0726-4bd8-a510-fbc1799b3eab-plugins-config-volume\") pod \"0233caa0-0726-4bd8-a510-fbc1799b3eab\" (UID: \"0233caa0-0726-4bd8-a510-fbc1799b3eab\") "
Nov 03 17:00:07.551929 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.551809    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0233caa0-0726-4bd8-a510-fbc1799b3eab-kube-api-access-bxp22\" (UniqueName: \"kubernetes.io/projected/0233caa0-0726-4bd8-a510-fbc1799b3eab-kube-api-access-bxp22\") pod \"0233caa0-0726-4bd8-a510-fbc1799b3eab\" (UID: \"0233caa0-0726-4bd8-a510-fbc1799b3eab\") "
Nov 03 17:00:07.552161 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.552134    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0233caa0-0726-4bd8-a510-fbc1799b3eab-plugins-config-volume" pod "0233caa0-0726-4bd8-a510-fbc1799b3eab" (UID: "0233caa0-0726-4bd8-a510-fbc1799b3eab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:00:07.552229 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.552171    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9vd8\" (UniqueName: \"kubernetes.io/projected/9744816f-937a-43ab-b49d-8cfc4c4522dc-kube-api-access-c9vd8\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:07.552229 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.552189    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9744816f-937a-43ab-b49d-8cfc4c4522dc-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:07.554078 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.554058    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0233caa0-0726-4bd8-a510-fbc1799b3eab-kube-api-access-bxp22" pod "0233caa0-0726-4bd8-a510-fbc1799b3eab" (UID: "0233caa0-0726-4bd8-a510-fbc1799b3eab"). InnerVolumeSpecName "kube-api-access-bxp22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:00:07.653096 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.653008    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0233caa0-0726-4bd8-a510-fbc1799b3eab-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:07.653096 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.653041    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxp22\" (UniqueName: \"kubernetes.io/projected/0233caa0-0726-4bd8-a510-fbc1799b3eab-kube-api-access-bxp22\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:07.861022 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.860985    5605 generic.go:356] "Generic (PLEG): container finished" podID="0233caa0-0726-4bd8-a510-fbc1799b3eab" containerID="5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688" exitCode=0
Nov 03 17:00:07.861212 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.861032    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"
Nov 03 17:00:07.861212 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.861070    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw" event={"ID":"0233caa0-0726-4bd8-a510-fbc1799b3eab","Type":"ContainerDied","Data":"5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688"}
Nov 03 17:00:07.861212 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.861156    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw" event={"ID":"0233caa0-0726-4bd8-a510-fbc1799b3eab","Type":"ContainerDied","Data":"5bc7d4f2836d983a62dd76d98e4bbb8cc6d9fae83c4292f7084cb25e8ad91f97"}
Nov 03 17:00:07.861212 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.861192    5605 scope.go:122] "RemoveContainer" containerID="5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688"
Nov 03 17:00:07.862637 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.862611    5605 generic.go:356] "Generic (PLEG): container finished" podID="9744816f-937a-43ab-b49d-8cfc4c4522dc" containerID="6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834" exitCode=0
Nov 03 17:00:07.862799 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.862643    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj" event={"ID":"9744816f-937a-43ab-b49d-8cfc4c4522dc","Type":"ContainerDied","Data":"6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834"}
Nov 03 17:00:07.862799 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.862667    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"
Nov 03 17:00:07.862799 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.862701    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj" event={"ID":"9744816f-937a-43ab-b49d-8cfc4c4522dc","Type":"ContainerDied","Data":"1369194df577cb48c9377bdca87097233f36e30a5aeedb1e8aaec9992e083d05"}
Nov 03 17:00:07.874320 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.874304    5605 scope.go:122] "RemoveContainer" containerID="5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688"
Nov 03 17:00:07.874587 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:07.874568    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688\": container with ID starting with 5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688 not found: ID does not exist" containerID="5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688"
Nov 03 17:00:07.874667 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.874595    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688"} err="failed to get container status \"5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688\": rpc error: code = NotFound desc = could not find container \"5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688\": container with ID starting with 5946e89315124abe89a9a31e025c14f8295b7f7185f8c5f350259f026d710688 not found: ID does not exist"
Nov 03 17:00:07.874667 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.874612    5605 scope.go:122] "RemoveContainer" containerID="6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834"
Nov 03 17:00:07.888464 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.888024    5605 scope.go:122] "RemoveContainer" containerID="6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834"
Nov 03 17:00:07.888464 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:07.888399    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834\": container with ID starting with 6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834 not found: ID does not exist" containerID="6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834"
Nov 03 17:00:07.888464 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:07.888428    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834"} err="failed to get container status \"6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834\": rpc error: code = NotFound desc = could not find container \"6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834\": container with ID starting with 6e5723e166468f605f4ad20f82d721c1c32267e5ff11628df291d28850f54834 not found: ID does not exist"
Nov 03 17:00:08.045514 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:08.045426    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"]
Nov 03 17:00:08.096926 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:08.096897    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d5937b-6fbb674d67-ws9xw"]
Nov 03 17:00:08.171567 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:08.171533    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"]
Nov 03 17:00:08.179821 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:08.179791    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-d5937b-dd5b499b5-rx4rj"]
Nov 03 17:00:08.308993 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:08.308915    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0233caa0-0726-4bd8-a510-fbc1799b3eab" path="/var/lib/kubelet/pods/0233caa0-0726-4bd8-a510-fbc1799b3eab/volumes"
Nov 03 17:00:08.309448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:08.309433    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9744816f-937a-43ab-b49d-8cfc4c4522dc" path="/var/lib/kubelet/pods/9744816f-937a-43ab-b49d-8cfc4c4522dc/volumes"
Nov 03 17:00:14.304427 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:14.304398    5605 scope.go:122] "RemoveContainer" containerID="3d1715bee3da4c9e11f5a870d81d8c669f921ad3281b60c6067eb88ede7e50d3"
Nov 03 17:00:14.304973 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:14.304637    5605 scope.go:122] "RemoveContainer" containerID="d2b14140f81438c44cdc186ea785b78914780f7f3b81beabdf43d17aab4bead8"
Nov 03 17:00:14.908283 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:14.908192    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" event={"ID":"b378df9b-dd3b-40c3-8886-1b67696431b3","Type":"ContainerStarted","Data":"bbb87a32b1fe3f706a53ed211b00b7445a62b1a2da95c442aa7f45a5799eece4"}
Nov 03 17:00:14.910162 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:14.910137    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" event={"ID":"30f6c764-7a1b-48e0-9b28-ddeb3b151f06","Type":"ContainerStarted","Data":"327c784805001ccd00f0863e7374cd7331dd78525ca31824d07cda025da26c56"}
Nov 03 17:00:15.151643 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.151588    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" podStartSLOduration=15.151572511 podStartE2EDuration="15.151572511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:00:15.150386204 +0000 UTC m=+2900.974837475" watchObservedRunningTime="2026-11-03 17:00:15.151572511 +0000 UTC m=+2900.976023781"
Nov 03 17:00:15.301661 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.301636    5605 scope.go:122] "RemoveContainer" containerID="eb724a38bcccca93ca6f219e1b247ba0aa900b05545316a5c9ace46b9ad04b10"
Nov 03 17:00:15.918781 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.918718    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw" event={"ID":"97bea05f-9f14-40e0-898a-b957b279858f","Type":"ContainerStarted","Data":"b7c21bdbf5cfb87d23ce2fcd2d53859f0e2f310e6da63a9c96cc99dea2a51b75"}
Nov 03 17:00:15.920620 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.920595    5605 generic.go:356] "Generic (PLEG): container finished" podID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerID="327c784805001ccd00f0863e7374cd7331dd78525ca31824d07cda025da26c56" exitCode=60
Nov 03 17:00:15.920779 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.920666    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" event={"ID":"30f6c764-7a1b-48e0-9b28-ddeb3b151f06","Type":"ContainerDied","Data":"327c784805001ccd00f0863e7374cd7331dd78525ca31824d07cda025da26c56"}
Nov 03 17:00:15.920779 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.920750    5605 scope.go:122] "RemoveContainer" containerID="3d1715bee3da4c9e11f5a870d81d8c669f921ad3281b60c6067eb88ede7e50d3"
Nov 03 17:00:15.921026 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.921010    5605 scope.go:122] "RemoveContainer" containerID="327c784805001ccd00f0863e7374cd7331dd78525ca31824d07cda025da26c56"
Nov 03 17:00:15.921305 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:15.921281    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc_odh-ai-gateway-infra(30f6c764-7a1b-48e0-9b28-ddeb3b151f06)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06"
Nov 03 17:00:15.922504 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.922484    5605 generic.go:356] "Generic (PLEG): container finished" podID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerID="bbb87a32b1fe3f706a53ed211b00b7445a62b1a2da95c442aa7f45a5799eece4" exitCode=60
Nov 03 17:00:15.922604 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.922513    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" event={"ID":"b378df9b-dd3b-40c3-8886-1b67696431b3","Type":"ContainerDied","Data":"bbb87a32b1fe3f706a53ed211b00b7445a62b1a2da95c442aa7f45a5799eece4"}
Nov 03 17:00:15.922802 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.922789    5605 scope.go:122] "RemoveContainer" containerID="bbb87a32b1fe3f706a53ed211b00b7445a62b1a2da95c442aa7f45a5799eece4"
Nov 03 17:00:15.922997 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:15.922981    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc_odh-ai-gateway-infra(b378df9b-dd3b-40c3-8886-1b67696431b3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3"
Nov 03 17:00:15.934235 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:15.934218    5605 scope.go:122] "RemoveContainer" containerID="d2b14140f81438c44cdc186ea785b78914780f7f3b81beabdf43d17aab4bead8"
Nov 03 17:00:16.047236 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:16.047201    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"]
Nov 03 17:00:16.097999 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:16.097950    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw" podStartSLOduration=16.097934312 podStartE2EDuration="16.097934312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:00:16.096010582 +0000 UTC m=+2901.920461854" watchObservedRunningTime="2026-11-03 17:00:16.097934312 +0000 UTC m=+2901.922385583"
Nov 03 17:00:16.164165 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:16.164134    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"]
Nov 03 17:00:16.931176 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:16.931145    5605 generic.go:356] "Generic (PLEG): container finished" podID="97bea05f-9f14-40e0-898a-b957b279858f" containerID="b7c21bdbf5cfb87d23ce2fcd2d53859f0e2f310e6da63a9c96cc99dea2a51b75" exitCode=60
Nov 03 17:00:16.931612 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:16.931225    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw" event={"ID":"97bea05f-9f14-40e0-898a-b957b279858f","Type":"ContainerDied","Data":"b7c21bdbf5cfb87d23ce2fcd2d53859f0e2f310e6da63a9c96cc99dea2a51b75"}
Nov 03 17:00:16.931612 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:16.931278    5605 scope.go:122] "RemoveContainer" containerID="eb724a38bcccca93ca6f219e1b247ba0aa900b05545316a5c9ace46b9ad04b10"
Nov 03 17:00:16.931744 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:16.931673    5605 scope.go:122] "RemoveContainer" containerID="b7c21bdbf5cfb87d23ce2fcd2d53859f0e2f310e6da63a9c96cc99dea2a51b75"
Nov 03 17:00:16.932065 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:16.932038    5605 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw_odh-ai-gateway-infra(97bea05f-9f14-40e0-898a-b957b279858f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw" podUID="97bea05f-9f14-40e0-898a-b957b279858f"
Nov 03 17:00:17.065590 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.065568    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"
Nov 03 17:00:17.073635 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.073612    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"]
Nov 03 17:00:17.107763 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.107742    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"
Nov 03 17:00:17.155426 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.155388    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b378df9b-dd3b-40c3-8886-1b67696431b3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b378df9b-dd3b-40c3-8886-1b67696431b3-maas-api-service-ca\") pod \"b378df9b-dd3b-40c3-8886-1b67696431b3\" (UID: \"b378df9b-dd3b-40c3-8886-1b67696431b3\") "
Nov 03 17:00:17.155623 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.155522    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b378df9b-dd3b-40c3-8886-1b67696431b3-kube-api-access-r2sbj\" (UniqueName: \"kubernetes.io/projected/b378df9b-dd3b-40c3-8886-1b67696431b3-kube-api-access-r2sbj\") pod \"b378df9b-dd3b-40c3-8886-1b67696431b3\" (UID: \"b378df9b-dd3b-40c3-8886-1b67696431b3\") "
Nov 03 17:00:17.155848 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.155824    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b378df9b-dd3b-40c3-8886-1b67696431b3-maas-api-service-ca" pod "b378df9b-dd3b-40c3-8886-1b67696431b3" (UID: "b378df9b-dd3b-40c3-8886-1b67696431b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:00:17.157817 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.157791    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b378df9b-dd3b-40c3-8886-1b67696431b3-kube-api-access-r2sbj" pod "b378df9b-dd3b-40c3-8886-1b67696431b3" (UID: "b378df9b-dd3b-40c3-8886-1b67696431b3"). InnerVolumeSpecName "kube-api-access-r2sbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:00:17.256761 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.256664    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-maas-api-service-ca\") pod \"30f6c764-7a1b-48e0-9b28-ddeb3b151f06\" (UID: \"30f6c764-7a1b-48e0-9b28-ddeb3b151f06\") "
Nov 03 17:00:17.256909 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.256763    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-kube-api-access-m6glc\" (UniqueName: \"kubernetes.io/projected/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-kube-api-access-m6glc\") pod \"30f6c764-7a1b-48e0-9b28-ddeb3b151f06\" (UID: \"30f6c764-7a1b-48e0-9b28-ddeb3b151f06\") "
Nov 03 17:00:17.257073 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.257056    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2sbj\" (UniqueName: \"kubernetes.io/projected/b378df9b-dd3b-40c3-8886-1b67696431b3-kube-api-access-r2sbj\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:17.257107 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.257079    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b378df9b-dd3b-40c3-8886-1b67696431b3-maas-api-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:17.257107 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.257087    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-maas-api-service-ca" pod "30f6c764-7a1b-48e0-9b28-ddeb3b151f06" (UID: "30f6c764-7a1b-48e0-9b28-ddeb3b151f06"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:00:17.258986 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.258965    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-kube-api-access-m6glc" pod "30f6c764-7a1b-48e0-9b28-ddeb3b151f06" (UID: "30f6c764-7a1b-48e0-9b28-ddeb3b151f06"). InnerVolumeSpecName "kube-api-access-m6glc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:00:17.358591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.358555    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-maas-api-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:17.358591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.358592    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6glc\" (UniqueName: \"kubernetes.io/projected/30f6c764-7a1b-48e0-9b28-ddeb3b151f06-kube-api-access-m6glc\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:17.939417 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.939388    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"
Nov 03 17:00:17.939885 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.939423    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc" event={"ID":"30f6c764-7a1b-48e0-9b28-ddeb3b151f06","Type":"ContainerDied","Data":"4b48bb03325b7cd67d8bb8095071cbf0919273cb5c7aa00c28f02dc5d8b0d650"}
Nov 03 17:00:17.939885 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.939502    5605 scope.go:122] "RemoveContainer" containerID="327c784805001ccd00f0863e7374cd7331dd78525ca31824d07cda025da26c56"
Nov 03 17:00:17.941474 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.941457    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"
Nov 03 17:00:17.941562 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.941484    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc" event={"ID":"b378df9b-dd3b-40c3-8886-1b67696431b3","Type":"ContainerDied","Data":"166a820fffe79a9f1e5a0b4b22d1566a6f5a6fb42091ef7ce017cdddf767848a"}
Nov 03 17:00:17.951633 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:17.951612    5605 scope.go:122] "RemoveContainer" containerID="bbb87a32b1fe3f706a53ed211b00b7445a62b1a2da95c442aa7f45a5799eece4"
Nov 03 17:00:18.090788 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.090764    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"
Nov 03 17:00:18.166082 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.166052    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97bea05f-9f14-40e0-898a-b957b279858f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bea05f-9f14-40e0-898a-b957b279858f-maas-api-service-ca\") pod \"97bea05f-9f14-40e0-898a-b957b279858f\" (UID: \"97bea05f-9f14-40e0-898a-b957b279858f\") "
Nov 03 17:00:18.166253 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.166156    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97bea05f-9f14-40e0-898a-b957b279858f-kube-api-access-plfs9\" (UniqueName: \"kubernetes.io/projected/97bea05f-9f14-40e0-898a-b957b279858f-kube-api-access-plfs9\") pod \"97bea05f-9f14-40e0-898a-b957b279858f\" (UID: \"97bea05f-9f14-40e0-898a-b957b279858f\") "
Nov 03 17:00:18.166401 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.166378    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97bea05f-9f14-40e0-898a-b957b279858f-maas-api-service-ca" pod "97bea05f-9f14-40e0-898a-b957b279858f" (UID: "97bea05f-9f14-40e0-898a-b957b279858f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:00:18.168420 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.168389    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97bea05f-9f14-40e0-898a-b957b279858f-kube-api-access-plfs9" pod "97bea05f-9f14-40e0-898a-b957b279858f" (UID: "97bea05f-9f14-40e0-898a-b957b279858f"). InnerVolumeSpecName "kube-api-access-plfs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:00:18.231332 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.231261    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"]
Nov 03 17:00:18.238938 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.238911    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-509f14-29895420-vh7gc"]
Nov 03 17:00:18.267861 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.267827    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bea05f-9f14-40e0-898a-b957b279858f-maas-api-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:18.267861 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.267855    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plfs9\" (UniqueName: \"kubernetes.io/projected/97bea05f-9f14-40e0-898a-b957b279858f-kube-api-access-plfs9\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:18.307278 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.307249    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3" path="/var/lib/kubelet/pods/b378df9b-dd3b-40c3-8886-1b67696431b3/volumes"
Nov 03 17:00:18.354363 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.354324    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"]
Nov 03 17:00:18.357040 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.357013    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-d5937b-29895420-2gcnc"]
Nov 03 17:00:18.952852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.952820    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"
Nov 03 17:00:18.953356 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.952814    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw" event={"ID":"97bea05f-9f14-40e0-898a-b957b279858f","Type":"ContainerDied","Data":"f8add702ac417f1866bd61363c60c01553c86c4659cf1ed571f16ca7f652012b"}
Nov 03 17:00:18.953356 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:18.952957    5605 scope.go:122] "RemoveContainer" containerID="b7c21bdbf5cfb87d23ce2fcd2d53859f0e2f310e6da63a9c96cc99dea2a51b75"
Nov 03 17:00:19.129774 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:19.127085    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"]
Nov 03 17:00:19.177371 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:19.177343    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-61be78-29895420-mb7dw"]
Nov 03 17:00:20.307699 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:20.307666    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" path="/var/lib/kubelet/pods/30f6c764-7a1b-48e0-9b28-ddeb3b151f06/volumes"
Nov 03 17:00:20.308106 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:20.308091    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97bea05f-9f14-40e0-898a-b957b279858f" path="/var/lib/kubelet/pods/97bea05f-9f14-40e0-898a-b957b279858f/volumes"
Nov 03 17:00:33.393707 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:33.393623    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:00:43.250781 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.250742    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"]
Nov 03 17:00:43.251243 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.250986    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8" podUID="c18b4cbe-f908-454c-bf78-612a5b64ae00" containerName="maas-api" containerID="cri-o://7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c" gracePeriod=30
Nov 03 17:00:43.521800 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.521774    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 17:00:43.630478 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.630445    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-maas-api-tls\") pod \"c18b4cbe-f908-454c-bf78-612a5b64ae00\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") "
Nov 03 17:00:43.630699 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.630504    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-metrics-certs\") pod \"c18b4cbe-f908-454c-bf78-612a5b64ae00\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") "
Nov 03 17:00:43.630699 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.630641    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c18b4cbe-f908-454c-bf78-612a5b64ae00-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c18b4cbe-f908-454c-bf78-612a5b64ae00-openshift-service-ca\") pod \"c18b4cbe-f908-454c-bf78-612a5b64ae00\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") "
Nov 03 17:00:43.630699 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.630660    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c18b4cbe-f908-454c-bf78-612a5b64ae00-kube-api-access-lz7zp\" (UniqueName: \"kubernetes.io/projected/c18b4cbe-f908-454c-bf78-612a5b64ae00-kube-api-access-lz7zp\") pod \"c18b4cbe-f908-454c-bf78-612a5b64ae00\" (UID: \"c18b4cbe-f908-454c-bf78-612a5b64ae00\") "
Nov 03 17:00:43.631100 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.631062    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c18b4cbe-f908-454c-bf78-612a5b64ae00-openshift-service-ca" pod "c18b4cbe-f908-454c-bf78-612a5b64ae00" (UID: "c18b4cbe-f908-454c-bf78-612a5b64ae00"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:00:43.633036 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.633005    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c18b4cbe-f908-454c-bf78-612a5b64ae00-kube-api-access-lz7zp" pod "c18b4cbe-f908-454c-bf78-612a5b64ae00" (UID: "c18b4cbe-f908-454c-bf78-612a5b64ae00"). InnerVolumeSpecName "kube-api-access-lz7zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:00:43.633160 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.632998    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-maas-api-tls" pod "c18b4cbe-f908-454c-bf78-612a5b64ae00" (UID: "c18b4cbe-f908-454c-bf78-612a5b64ae00"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:00:43.633160 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.633043    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-metrics-certs" pod "c18b4cbe-f908-454c-bf78-612a5b64ae00" (UID: "c18b4cbe-f908-454c-bf78-612a5b64ae00"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:00:43.731697 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.731658    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:43.731697 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.731693    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c18b4cbe-f908-454c-bf78-612a5b64ae00-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:43.731697 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.731703    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz7zp\" (UniqueName: \"kubernetes.io/projected/c18b4cbe-f908-454c-bf78-612a5b64ae00-kube-api-access-lz7zp\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:43.731985 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:43.731712    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18b4cbe-f908-454c-bf78-612a5b64ae00-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:44.077640 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:44.077601    5605 generic.go:356] "Generic (PLEG): container finished" podID="c18b4cbe-f908-454c-bf78-612a5b64ae00" containerID="7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c" exitCode=0
Nov 03 17:00:44.077849 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:44.077647    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"
Nov 03 17:00:44.077849 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:44.077689    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8" event={"ID":"c18b4cbe-f908-454c-bf78-612a5b64ae00","Type":"ContainerDied","Data":"7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c"}
Nov 03 17:00:44.077849 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:44.077775    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8" event={"ID":"c18b4cbe-f908-454c-bf78-612a5b64ae00","Type":"ContainerDied","Data":"2a90b9ce7e4a992a50c28b843aa0b09e1b3f2bc8dbb4da2b448e2000985adb13"}
Nov 03 17:00:44.077849 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:44.077800    5605 scope.go:122] "RemoveContainer" containerID="7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c"
Nov 03 17:00:44.087315 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:44.087296    5605 scope.go:122] "RemoveContainer" containerID="7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c"
Nov 03 17:00:44.087586 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:44.087567    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c\": container with ID starting with 7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c not found: ID does not exist" containerID="7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c"
Nov 03 17:00:44.087631 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:44.087595    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c"} err="failed to get container status \"7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c\": rpc error: code = NotFound desc = could not find container \"7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c\": container with ID starting with 7add8496c1a8c9eee2ea6bd9e5766d0ff1bfbb4edeebedb6091ce09517c38f4c not found: ID does not exist"
Nov 03 17:00:44.249828 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:44.249795    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"]
Nov 03 17:00:44.300612 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:44.300585    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-d5937b-744cc9d5c4-mftz8"]
Nov 03 17:00:44.307061 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:44.307038    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c18b4cbe-f908-454c-bf78-612a5b64ae00" path="/var/lib/kubelet/pods/c18b4cbe-f908-454c-bf78-612a5b64ae00/volumes"
Nov 03 17:00:53.149741 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:53.149690    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"]
Nov 03 17:00:53.150298 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:53.150006    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy" containerID="cri-o://be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35" gracePeriod=30
Nov 03 17:00:58.407086 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.407051    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 17:00:58.502272 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502178    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-kube-api-access-28dxn\" (UniqueName: \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-kube-api-access-28dxn\") pod \"41871919-9219-49c2-abd8-6a38e7e32d2e\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") "
Nov 03 17:00:58.502429 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502279    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-token\" (UniqueName: \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-token\") pod \"41871919-9219-49c2-abd8-6a38e7e32d2e\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") "
Nov 03 17:00:58.502429 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502349    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-credential-socket\") pod \"41871919-9219-49c2-abd8-6a38e7e32d2e\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") "
Nov 03 17:00:58.502429 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502375    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-socket\") pod \"41871919-9219-49c2-abd8-6a38e7e32d2e\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") "
Nov 03 17:00:58.502429 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502408    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-data\") pod \"41871919-9219-49c2-abd8-6a38e7e32d2e\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") "
Nov 03 17:00:58.502621 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502436    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-podinfo\") pod \"41871919-9219-49c2-abd8-6a38e7e32d2e\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") "
Nov 03 17:00:58.502621 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502491    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-certs\") pod \"41871919-9219-49c2-abd8-6a38e7e32d2e\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") "
Nov 03 17:00:58.502621 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502520    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-envoy\") pod \"41871919-9219-49c2-abd8-6a38e7e32d2e\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") "
Nov 03 17:00:58.502621 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502578    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41871919-9219-49c2-abd8-6a38e7e32d2e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41871919-9219-49c2-abd8-6a38e7e32d2e-istiod-ca-cert\") pod \"41871919-9219-49c2-abd8-6a38e7e32d2e\" (UID: \"41871919-9219-49c2-abd8-6a38e7e32d2e\") "
Nov 03 17:00:58.502856 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502668    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-credential-socket" pod "41871919-9219-49c2-abd8-6a38e7e32d2e" (UID: "41871919-9219-49c2-abd8-6a38e7e32d2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:00:58.502856 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502680    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-socket" pod "41871919-9219-49c2-abd8-6a38e7e32d2e" (UID: "41871919-9219-49c2-abd8-6a38e7e32d2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:00:58.502856 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502768    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-data" pod "41871919-9219-49c2-abd8-6a38e7e32d2e" (UID: "41871919-9219-49c2-abd8-6a38e7e32d2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:00:58.503005 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.502881    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-certs" pod "41871919-9219-49c2-abd8-6a38e7e32d2e" (UID: "41871919-9219-49c2-abd8-6a38e7e32d2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:00:58.503269 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.503244    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41871919-9219-49c2-abd8-6a38e7e32d2e-istiod-ca-cert" pod "41871919-9219-49c2-abd8-6a38e7e32d2e" (UID: "41871919-9219-49c2-abd8-6a38e7e32d2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:00:58.503529 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.503326    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:58.503529 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.503352    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:58.503529 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.503368    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:58.503529 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.503383    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:58.504784 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.504699    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-kube-api-access-28dxn" pod "41871919-9219-49c2-abd8-6a38e7e32d2e" (UID: "41871919-9219-49c2-abd8-6a38e7e32d2e"). InnerVolumeSpecName "kube-api-access-28dxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:00:58.504784 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.504707    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-token" pod "41871919-9219-49c2-abd8-6a38e7e32d2e" (UID: "41871919-9219-49c2-abd8-6a38e7e32d2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:00:58.504957 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.504904    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-podinfo" pod "41871919-9219-49c2-abd8-6a38e7e32d2e" (UID: "41871919-9219-49c2-abd8-6a38e7e32d2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 17:00:58.505017 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.504974    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-envoy" pod "41871919-9219-49c2-abd8-6a38e7e32d2e" (UID: "41871919-9219-49c2-abd8-6a38e7e32d2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:00:58.604830 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.604796    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28dxn\" (UniqueName: \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-kube-api-access-28dxn\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:58.604830 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.604822    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:58.604830 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.604832    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:58.605120 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.604841    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41871919-9219-49c2-abd8-6a38e7e32d2e-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:58.605120 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:58.604850    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41871919-9219-49c2-abd8-6a38e7e32d2e-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:00:59.155590 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.155554    5605 generic.go:356] "Generic (PLEG): container finished" podID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerID="be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35" exitCode=0
Nov 03 17:00:59.155796 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.155602    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" event={"ID":"41871919-9219-49c2-abd8-6a38e7e32d2e","Type":"ContainerDied","Data":"be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35"}
Nov 03 17:00:59.155796 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.155617    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"
Nov 03 17:00:59.155796 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.155644    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f" event={"ID":"41871919-9219-49c2-abd8-6a38e7e32d2e","Type":"ContainerDied","Data":"289e205c26cb6346f39319d3ee442961754a805bdf6a6dd0547f7ecd301ed107"}
Nov 03 17:00:59.155796 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.155666    5605 scope.go:122] "RemoveContainer" containerID="be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35"
Nov 03 17:00:59.166123 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.166098    5605 scope.go:122] "RemoveContainer" containerID="e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679"
Nov 03 17:00:59.175705 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.175678    5605 scope.go:122] "RemoveContainer" containerID="be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35"
Nov 03 17:00:59.176089 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:59.176064    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35\": container with ID starting with be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35 not found: ID does not exist" containerID="be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35"
Nov 03 17:00:59.176152 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.176098    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35"} err="failed to get container status \"be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35\": rpc error: code = NotFound desc = could not find container \"be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35\": container with ID starting with be2b6f9e9be01710c5c7cd03c2ed011e56439e8f2a3ba1fba7ca7a614b31ef35 not found: ID does not exist"
Nov 03 17:00:59.176152 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.176119    5605 scope.go:122] "RemoveContainer" containerID="e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679"
Nov 03 17:00:59.176391 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:00:59.176368    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679\": container with ID starting with e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679 not found: ID does not exist" containerID="e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679"
Nov 03 17:00:59.176456 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.176402    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679"} err="failed to get container status \"e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679\": rpc error: code = NotFound desc = could not find container \"e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679\": container with ID starting with e0dff3f28df63f445223835f262aafafe11183173173b4f926936c6401282679 not found: ID does not exist"
Nov 03 17:00:59.380745 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.380688    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"]
Nov 03 17:00:59.432933 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:00:59.432851    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-d5937b-openshift-default-7f69cbc74c-jcj7f"]
Nov 03 17:01:00.308418 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:00.308381    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" path="/var/lib/kubelet/pods/41871919-9219-49c2-abd8-6a38e7e32d2e/volumes"
Nov 03 17:01:02.006632 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:02.006595    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"]
Nov 03 17:01:03.392283 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:03.392246    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:01:05.656890 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:05.656855    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"]
Nov 03 17:01:05.760866 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:05.760823    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"]
Nov 03 17:01:10.657420 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:10.657356    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl" podUID="b058295c-3fa6-409d-83ee-981d4ee60381" containerName="payload-processing" containerID="cri-o://5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e" gracePeriod=25
Nov 03 17:01:10.761603 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:10.761527    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j" podUID="626e2175-2d9f-483c-92bd-f3538bd95f22" containerName="payload-pre-processing" containerID="cri-o://e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60" gracePeriod=25
Nov 03 17:01:10.937824 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:10.937786    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 17:01:11.033437 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.033413    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 17:01:11.036646 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.036629    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/626e2175-2d9f-483c-92bd-f3538bd95f22-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/626e2175-2d9f-483c-92bd-f3538bd95f22-plugins-config-volume\") pod \"626e2175-2d9f-483c-92bd-f3538bd95f22\" (UID: \"626e2175-2d9f-483c-92bd-f3538bd95f22\") "
Nov 03 17:01:11.036713 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.036674    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b058295c-3fa6-409d-83ee-981d4ee60381-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b058295c-3fa6-409d-83ee-981d4ee60381-plugins-config-volume\") pod \"b058295c-3fa6-409d-83ee-981d4ee60381\" (UID: \"b058295c-3fa6-409d-83ee-981d4ee60381\") "
Nov 03 17:01:11.036713 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.036695    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/626e2175-2d9f-483c-92bd-f3538bd95f22-kube-api-access-ddnd4\" (UniqueName: \"kubernetes.io/projected/626e2175-2d9f-483c-92bd-f3538bd95f22-kube-api-access-ddnd4\") pod \"626e2175-2d9f-483c-92bd-f3538bd95f22\" (UID: \"626e2175-2d9f-483c-92bd-f3538bd95f22\") "
Nov 03 17:01:11.036823 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.036761    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b058295c-3fa6-409d-83ee-981d4ee60381-kube-api-access-g882r\" (UniqueName: \"kubernetes.io/projected/b058295c-3fa6-409d-83ee-981d4ee60381-kube-api-access-g882r\") pod \"b058295c-3fa6-409d-83ee-981d4ee60381\" (UID: \"b058295c-3fa6-409d-83ee-981d4ee60381\") "
Nov 03 17:01:11.037043 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.037019    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/626e2175-2d9f-483c-92bd-f3538bd95f22-plugins-config-volume" pod "626e2175-2d9f-483c-92bd-f3538bd95f22" (UID: "626e2175-2d9f-483c-92bd-f3538bd95f22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:01:11.037128 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.037086    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b058295c-3fa6-409d-83ee-981d4ee60381-plugins-config-volume" pod "b058295c-3fa6-409d-83ee-981d4ee60381" (UID: "b058295c-3fa6-409d-83ee-981d4ee60381"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:01:11.038952 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.038933    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/626e2175-2d9f-483c-92bd-f3538bd95f22-kube-api-access-ddnd4" pod "626e2175-2d9f-483c-92bd-f3538bd95f22" (UID: "626e2175-2d9f-483c-92bd-f3538bd95f22"). InnerVolumeSpecName "kube-api-access-ddnd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:01:11.039030 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.038987    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b058295c-3fa6-409d-83ee-981d4ee60381-kube-api-access-g882r" pod "b058295c-3fa6-409d-83ee-981d4ee60381" (UID: "b058295c-3fa6-409d-83ee-981d4ee60381"). InnerVolumeSpecName "kube-api-access-g882r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:01:11.138075 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.138033    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/626e2175-2d9f-483c-92bd-f3538bd95f22-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:11.138075 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.138064    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b058295c-3fa6-409d-83ee-981d4ee60381-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:11.138075 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.138074    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddnd4\" (UniqueName: \"kubernetes.io/projected/626e2175-2d9f-483c-92bd-f3538bd95f22-kube-api-access-ddnd4\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:11.138075 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.138083    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g882r\" (UniqueName: \"kubernetes.io/projected/b058295c-3fa6-409d-83ee-981d4ee60381-kube-api-access-g882r\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:11.216918 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.216825    5605 generic.go:356] "Generic (PLEG): container finished" podID="b058295c-3fa6-409d-83ee-981d4ee60381" containerID="5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e" exitCode=0
Nov 03 17:01:11.216918 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.216884    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"
Nov 03 17:01:11.216918 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.216905    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl" event={"ID":"b058295c-3fa6-409d-83ee-981d4ee60381","Type":"ContainerDied","Data":"5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e"}
Nov 03 17:01:11.217209 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.216997    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl" event={"ID":"b058295c-3fa6-409d-83ee-981d4ee60381","Type":"ContainerDied","Data":"c421d40657d6dfafe91a2ad432d79dfed3a9c146fafa0901deaed7b217725159"}
Nov 03 17:01:11.217209 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.217050    5605 scope.go:122] "RemoveContainer" containerID="5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e"
Nov 03 17:01:11.218665 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.218620    5605 generic.go:356] "Generic (PLEG): container finished" podID="626e2175-2d9f-483c-92bd-f3538bd95f22" containerID="e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60" exitCode=0
Nov 03 17:01:11.218811 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.218685    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"
Nov 03 17:01:11.218811 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.218696    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j" event={"ID":"626e2175-2d9f-483c-92bd-f3538bd95f22","Type":"ContainerDied","Data":"e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60"}
Nov 03 17:01:11.218811 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.218778    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j" event={"ID":"626e2175-2d9f-483c-92bd-f3538bd95f22","Type":"ContainerDied","Data":"abbbccedd05f03c8610dfadaf0dc963de9e5d8eca610f5b9bd39dc4c0b49d9c9"}
Nov 03 17:01:11.226157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.226138    5605 scope.go:122] "RemoveContainer" containerID="5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e"
Nov 03 17:01:11.226426 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:11.226407    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e\": container with ID starting with 5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e not found: ID does not exist" containerID="5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e"
Nov 03 17:01:11.226514 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.226437    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e"} err="failed to get container status \"5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e\": rpc error: code = NotFound desc = could not find container \"5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e\": container with ID starting with 5b244d754960986c749edff58b4539dceb5de92d09340d7da15c0e6a8647e32e not found: ID does not exist"
Nov 03 17:01:11.226514 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.226461    5605 scope.go:122] "RemoveContainer" containerID="e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60"
Nov 03 17:01:11.235103 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.235087    5605 scope.go:122] "RemoveContainer" containerID="e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60"
Nov 03 17:01:11.235379 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:11.235358    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60\": container with ID starting with e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60 not found: ID does not exist" containerID="e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60"
Nov 03 17:01:11.235426 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.235388    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60"} err="failed to get container status \"e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60\": rpc error: code = NotFound desc = could not find container \"e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60\": container with ID starting with e1f23f4249ad3f08b980b8595d53ef395b10e92de9d401cfbaebf999e1600a60 not found: ID does not exist"
Nov 03 17:01:11.388918 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.388883    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"]
Nov 03 17:01:11.395470 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.395439    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-195095-77757dc457-zrrrl"]
Nov 03 17:01:11.510506 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.510416    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"]
Nov 03 17:01:11.516646 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:11.516616    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-195095-8574b8d94c-f8n7j"]
Nov 03 17:01:12.307541 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:12.307505    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="626e2175-2d9f-483c-92bd-f3538bd95f22" path="/var/lib/kubelet/pods/626e2175-2d9f-483c-92bd-f3538bd95f22/volumes"
Nov 03 17:01:12.308014 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:12.307945    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b058295c-3fa6-409d-83ee-981d4ee60381" path="/var/lib/kubelet/pods/b058295c-3fa6-409d-83ee-981d4ee60381/volumes"
Nov 03 17:01:17.027456 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.027416    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main" containerID="cri-o://15aa712ca4820c14f48e19d426977909081ba713f230b595cd07447b5a952fda" gracePeriod=45
Nov 03 17:01:17.251282 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.251249    5605 generic.go:356] "Generic (PLEG): container finished" podID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerID="15aa712ca4820c14f48e19d426977909081ba713f230b595cd07447b5a952fda" exitCode=0
Nov 03 17:01:17.251467 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.251326    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" event={"ID":"6376ccbd-aea0-41b3-bd45-cdbe2f999e28","Type":"ContainerDied","Data":"15aa712ca4820c14f48e19d426977909081ba713f230b595cd07447b5a952fda"}
Nov 03 17:01:17.282657 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.282596    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 17:01:17.309513 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.309482    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-home\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-home\") pod \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") "
Nov 03 17:01:17.309680 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.309552    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-kube-api-access-kf2zt\" (UniqueName: \"kubernetes.io/projected/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-kube-api-access-kf2zt\") pod \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") "
Nov 03 17:01:17.309680 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.309572    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-model-cache\") pod \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") "
Nov 03 17:01:17.309680 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.309621    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-dshm\") pod \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") "
Nov 03 17:01:17.309871 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.309685    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tls-certs\" (UniqueName: \"kubernetes.io/secret/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tls-certs\") pod \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") "
Nov 03 17:01:17.309871 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.309760    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tmp-dir\") pod \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\" (UID: \"6376ccbd-aea0-41b3-bd45-cdbe2f999e28\") "
Nov 03 17:01:17.309871 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.309833    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-home" pod "6376ccbd-aea0-41b3-bd45-cdbe2f999e28" (UID: "6376ccbd-aea0-41b3-bd45-cdbe2f999e28"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:17.310037 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.309892    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-model-cache" pod "6376ccbd-aea0-41b3-bd45-cdbe2f999e28" (UID: "6376ccbd-aea0-41b3-bd45-cdbe2f999e28"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:17.310095 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.310080    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tmp-dir" pod "6376ccbd-aea0-41b3-bd45-cdbe2f999e28" (UID: "6376ccbd-aea0-41b3-bd45-cdbe2f999e28"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:17.310205 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.310188    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:17.310271 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.310208    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:17.310271 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.310217    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:17.312153 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.312129    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tls-certs" pod "6376ccbd-aea0-41b3-bd45-cdbe2f999e28" (UID: "6376ccbd-aea0-41b3-bd45-cdbe2f999e28"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:01:17.312337 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.312323    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-kube-api-access-kf2zt" pod "6376ccbd-aea0-41b3-bd45-cdbe2f999e28" (UID: "6376ccbd-aea0-41b3-bd45-cdbe2f999e28"). InnerVolumeSpecName "kube-api-access-kf2zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:01:17.312406 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.312334    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-dshm" pod "6376ccbd-aea0-41b3-bd45-cdbe2f999e28" (UID: "6376ccbd-aea0-41b3-bd45-cdbe2f999e28"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:17.411296 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.411263    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:17.411296 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.411294    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf2zt\" (UniqueName: \"kubernetes.io/projected/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-kube-api-access-kf2zt\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:17.411570 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.411310    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6376ccbd-aea0-41b3-bd45-cdbe2f999e28-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:17.621446 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:17.621413    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"]
Nov 03 17:01:18.257309 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:18.257276    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"
Nov 03 17:01:18.257309 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:18.257305    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv" event={"ID":"6376ccbd-aea0-41b3-bd45-cdbe2f999e28","Type":"ContainerDied","Data":"7984c37f33451e0e5d3105bc4b833e5ab9610295451a9dba6c7366de3d7e0333"}
Nov 03 17:01:18.257840 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:18.257350    5605 scope.go:122] "RemoveContainer" containerID="15aa712ca4820c14f48e19d426977909081ba713f230b595cd07447b5a952fda"
Nov 03 17:01:18.431870 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:18.431835    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"]
Nov 03 17:01:18.441121 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:18.441091    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-bcd756/distinct-2-w1-bcd756-kserve-5b5f654846-ndwsv"]
Nov 03 17:01:20.306999 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:20.306970    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" path="/var/lib/kubelet/pods/6376ccbd-aea0-41b3-bd45-cdbe2f999e28/volumes"
Nov 03 17:01:21.694889 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:21.694856    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"]
Nov 03 17:01:21.695286 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:21.695085    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8" podUID="b9df6a16-4e47-4794-a2be-cd38093531bb" containerName="maas-api" containerID="cri-o://37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680" gracePeriod=30
Nov 03 17:01:21.858429 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:21.858385    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8" podUID="b9df6a16-4e47-4794-a2be-cd38093531bb" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.119:8443/health\": dial tcp 10.133.0.119:8443: connect: connection refused"
Nov 03 17:01:21.950250 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:21.950185    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 17:01:22.060746 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.060271    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-maas-api-tls\") pod \"b9df6a16-4e47-4794-a2be-cd38093531bb\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") "
Nov 03 17:01:22.060746 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.060455    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-metrics-certs\") pod \"b9df6a16-4e47-4794-a2be-cd38093531bb\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") "
Nov 03 17:01:22.060746 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.060543    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9df6a16-4e47-4794-a2be-cd38093531bb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9df6a16-4e47-4794-a2be-cd38093531bb-openshift-service-ca\") pod \"b9df6a16-4e47-4794-a2be-cd38093531bb\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") "
Nov 03 17:01:22.060746 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.060594    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9df6a16-4e47-4794-a2be-cd38093531bb-kube-api-access-5w22w\" (UniqueName: \"kubernetes.io/projected/b9df6a16-4e47-4794-a2be-cd38093531bb-kube-api-access-5w22w\") pod \"b9df6a16-4e47-4794-a2be-cd38093531bb\" (UID: \"b9df6a16-4e47-4794-a2be-cd38093531bb\") "
Nov 03 17:01:22.062174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.062087    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9df6a16-4e47-4794-a2be-cd38093531bb-openshift-service-ca" pod "b9df6a16-4e47-4794-a2be-cd38093531bb" (UID: "b9df6a16-4e47-4794-a2be-cd38093531bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:01:22.067895 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.067857    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-maas-api-tls" pod "b9df6a16-4e47-4794-a2be-cd38093531bb" (UID: "b9df6a16-4e47-4794-a2be-cd38093531bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:01:22.067895 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.067877    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-metrics-certs" pod "b9df6a16-4e47-4794-a2be-cd38093531bb" (UID: "b9df6a16-4e47-4794-a2be-cd38093531bb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:01:22.068113 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.067969    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9df6a16-4e47-4794-a2be-cd38093531bb-kube-api-access-5w22w" pod "b9df6a16-4e47-4794-a2be-cd38093531bb" (UID: "b9df6a16-4e47-4794-a2be-cd38093531bb"). InnerVolumeSpecName "kube-api-access-5w22w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:01:22.161498 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.161466    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:22.161498 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.161496    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9df6a16-4e47-4794-a2be-cd38093531bb-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:22.161498 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.161506    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w22w\" (UniqueName: \"kubernetes.io/projected/b9df6a16-4e47-4794-a2be-cd38093531bb-kube-api-access-5w22w\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:22.161850 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.161517    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9df6a16-4e47-4794-a2be-cd38093531bb-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:22.277298 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.277209    5605 generic.go:356] "Generic (PLEG): container finished" podID="b9df6a16-4e47-4794-a2be-cd38093531bb" containerID="37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680" exitCode=0
Nov 03 17:01:22.277298 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.277255    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"
Nov 03 17:01:22.277298 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.277260    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8" event={"ID":"b9df6a16-4e47-4794-a2be-cd38093531bb","Type":"ContainerDied","Data":"37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680"}
Nov 03 17:01:22.277586 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.277340    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8" event={"ID":"b9df6a16-4e47-4794-a2be-cd38093531bb","Type":"ContainerDied","Data":"4203a5471753ee5d952c47913445850b387255c00fe76dc02500d104e4b141e7"}
Nov 03 17:01:22.277586 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.277371    5605 scope.go:122] "RemoveContainer" containerID="37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680"
Nov 03 17:01:22.287382 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.287365    5605 scope.go:122] "RemoveContainer" containerID="37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680"
Nov 03 17:01:22.287640 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:22.287620    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680\": container with ID starting with 37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680 not found: ID does not exist" containerID="37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680"
Nov 03 17:01:22.287743 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.287649    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680"} err="failed to get container status \"37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680\": rpc error: code = NotFound desc = could not find container \"37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680\": container with ID starting with 37b86a89eb5b1e3b457126af9c4e7587349deb7e005fa8f7945749cebb575680 not found: ID does not exist"
Nov 03 17:01:22.448977 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.448937    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"]
Nov 03 17:01:22.456625 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.456595    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-195095-746969d4-w4qj8"]
Nov 03 17:01:22.622659 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.622619    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf" podUID="22d32ce6-2fd8-49b4-8ade-ab2c12b24e44" containerName="payload-processing" containerID="cri-o://8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d" gracePeriod=25
Nov 03 17:01:22.873129 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.873070    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 17:01:22.974272 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.974244    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-kube-api-access-8b449\" (UniqueName: \"kubernetes.io/projected/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-kube-api-access-8b449\") pod \"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44\" (UID: \"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44\") "
Nov 03 17:01:22.974422 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.974332    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-plugins-config-volume\") pod \"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44\" (UID: \"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44\") "
Nov 03 17:01:22.974741 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.974706    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-plugins-config-volume" pod "22d32ce6-2fd8-49b4-8ade-ab2c12b24e44" (UID: "22d32ce6-2fd8-49b4-8ade-ab2c12b24e44"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:01:22.976585 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:22.976563    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-kube-api-access-8b449" pod "22d32ce6-2fd8-49b4-8ade-ab2c12b24e44" (UID: "22d32ce6-2fd8-49b4-8ade-ab2c12b24e44"). InnerVolumeSpecName "kube-api-access-8b449". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:01:23.076059 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.076028    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b449\" (UniqueName: \"kubernetes.io/projected/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-kube-api-access-8b449\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:23.076059 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.076055    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:23.284499 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.284407    5605 generic.go:356] "Generic (PLEG): container finished" podID="22d32ce6-2fd8-49b4-8ade-ab2c12b24e44" containerID="8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d" exitCode=0
Nov 03 17:01:23.284499 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.284454    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"
Nov 03 17:01:23.284499 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.284491    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf" event={"ID":"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44","Type":"ContainerDied","Data":"8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d"}
Nov 03 17:01:23.284843 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.284537    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf" event={"ID":"22d32ce6-2fd8-49b4-8ade-ab2c12b24e44","Type":"ContainerDied","Data":"019d541dde5058ba3e722a014b65727d5d481509432d0a6d307be495d27d7b92"}
Nov 03 17:01:23.284843 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.284558    5605 scope.go:122] "RemoveContainer" containerID="8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d"
Nov 03 17:01:23.293718 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.293703    5605 scope.go:122] "RemoveContainer" containerID="8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d"
Nov 03 17:01:23.294010 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:23.293989    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d\": container with ID starting with 8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d not found: ID does not exist" containerID="8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d"
Nov 03 17:01:23.294056 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.294019    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d"} err="failed to get container status \"8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d\": rpc error: code = NotFound desc = could not find container \"8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d\": container with ID starting with 8274d5babf41d956f8772f85082e3ee08d93dee92a369275ae4a7e2f14f39b8d not found: ID does not exist"
Nov 03 17:01:23.459750 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.459691    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"]
Nov 03 17:01:23.470825 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:23.470793    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-963a2f-8dd66545-jqcvf"]
Nov 03 17:01:24.307649 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:24.307618    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22d32ce6-2fd8-49b4-8ade-ab2c12b24e44" path="/var/lib/kubelet/pods/22d32ce6-2fd8-49b4-8ade-ab2c12b24e44/volumes"
Nov 03 17:01:24.308149 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:24.308052    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9df6a16-4e47-4794-a2be-cd38093531bb" path="/var/lib/kubelet/pods/b9df6a16-4e47-4794-a2be-cd38093531bb/volumes"
Nov 03 17:01:33.251696 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:33.251659    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"]
Nov 03 17:01:33.252110 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:33.251947    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy" containerID="cri-o://c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66" gracePeriod=30
Nov 03 17:01:33.407807 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:33.407767    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:01:36.591658 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:36.591622    5605 patch_prober.go:29] interesting pod/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 17:01:36.592087 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:36.591684    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 17:01:38.428094 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.428055    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"]
Nov 03 17:01:38.428535 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.428311    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy" containerID="cri-o://82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13" gracePeriod=30
Nov 03 17:01:38.540529 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.540476    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 17:01:38.632060 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632021    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-credential-socket\") pod \"0c75fa68-0957-49ae-a013-0f8fae42dd31\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") "
Nov 03 17:01:38.632265 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632075    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-certs\") pod \"0c75fa68-0957-49ae-a013-0f8fae42dd31\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") "
Nov 03 17:01:38.632265 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632100    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-socket\") pod \"0c75fa68-0957-49ae-a013-0f8fae42dd31\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") "
Nov 03 17:01:38.632265 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632124    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-data\") pod \"0c75fa68-0957-49ae-a013-0f8fae42dd31\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") "
Nov 03 17:01:38.632265 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632203    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-envoy\") pod \"0c75fa68-0957-49ae-a013-0f8fae42dd31\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") "
Nov 03 17:01:38.632479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632275    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c75fa68-0957-49ae-a013-0f8fae42dd31-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c75fa68-0957-49ae-a013-0f8fae42dd31-istiod-ca-cert\") pod \"0c75fa68-0957-49ae-a013-0f8fae42dd31\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") "
Nov 03 17:01:38.632479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632308    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-kube-api-access-mwf82\" (UniqueName: \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-kube-api-access-mwf82\") pod \"0c75fa68-0957-49ae-a013-0f8fae42dd31\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") "
Nov 03 17:01:38.632479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632349    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-token\" (UniqueName: \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-token\") pod \"0c75fa68-0957-49ae-a013-0f8fae42dd31\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") "
Nov 03 17:01:38.632479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632365    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-credential-socket" pod "0c75fa68-0957-49ae-a013-0f8fae42dd31" (UID: "0c75fa68-0957-49ae-a013-0f8fae42dd31"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:38.632479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632410    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-podinfo\") pod \"0c75fa68-0957-49ae-a013-0f8fae42dd31\" (UID: \"0c75fa68-0957-49ae-a013-0f8fae42dd31\") "
Nov 03 17:01:38.632766 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632481    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-certs" pod "0c75fa68-0957-49ae-a013-0f8fae42dd31" (UID: "0c75fa68-0957-49ae-a013-0f8fae42dd31"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:38.632827 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632757    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c75fa68-0957-49ae-a013-0f8fae42dd31-istiod-ca-cert" pod "0c75fa68-0957-49ae-a013-0f8fae42dd31" (UID: "0c75fa68-0957-49ae-a013-0f8fae42dd31"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:01:38.632886 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632852    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c75fa68-0957-49ae-a013-0f8fae42dd31-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:38.632940 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632880    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:38.632940 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632899    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:38.633045 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.632992    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-socket" pod "0c75fa68-0957-49ae-a013-0f8fae42dd31" (UID: "0c75fa68-0957-49ae-a013-0f8fae42dd31"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:38.633099 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.633056    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-data" pod "0c75fa68-0957-49ae-a013-0f8fae42dd31" (UID: "0c75fa68-0957-49ae-a013-0f8fae42dd31"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:38.634744 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.634710    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-kube-api-access-mwf82" pod "0c75fa68-0957-49ae-a013-0f8fae42dd31" (UID: "0c75fa68-0957-49ae-a013-0f8fae42dd31"). InnerVolumeSpecName "kube-api-access-mwf82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:01:38.635213 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.634951    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-token" pod "0c75fa68-0957-49ae-a013-0f8fae42dd31" (UID: "0c75fa68-0957-49ae-a013-0f8fae42dd31"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:01:38.635213 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.635124    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-envoy" pod "0c75fa68-0957-49ae-a013-0f8fae42dd31" (UID: "0c75fa68-0957-49ae-a013-0f8fae42dd31"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:38.635355 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.635230    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-podinfo" pod "0c75fa68-0957-49ae-a013-0f8fae42dd31" (UID: "0c75fa68-0957-49ae-a013-0f8fae42dd31"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 17:01:38.733972 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.733935    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:38.733972 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.733966    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:38.733972 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.733975    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:38.734225 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.733985    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwf82\" (UniqueName: \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-kube-api-access-mwf82\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:38.734225 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.733996    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:38.734225 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:38.734004    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c75fa68-0957-49ae-a013-0f8fae42dd31-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:39.366409 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.366376    5605 generic.go:356] "Generic (PLEG): container finished" podID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerID="c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66" exitCode=0
Nov 03 17:01:39.366606 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.366434    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"
Nov 03 17:01:39.366606 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.366462    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" event={"ID":"0c75fa68-0957-49ae-a013-0f8fae42dd31","Type":"ContainerDied","Data":"c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66"}
Nov 03 17:01:39.366606 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.366521    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r" event={"ID":"0c75fa68-0957-49ae-a013-0f8fae42dd31","Type":"ContainerDied","Data":"c1d51d236de0201737b4bb43d2ce1d9b3f0bc9633769a9c7c9dd4819bd6aa067"}
Nov 03 17:01:39.366606 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.366544    5605 scope.go:122] "RemoveContainer" containerID="c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66"
Nov 03 17:01:39.376150 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.376131    5605 scope.go:122] "RemoveContainer" containerID="5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c"
Nov 03 17:01:39.384585 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.384558    5605 scope.go:122] "RemoveContainer" containerID="c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66"
Nov 03 17:01:39.385001 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:39.384981    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66\": container with ID starting with c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66 not found: ID does not exist" containerID="c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66"
Nov 03 17:01:39.385085 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.385016    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66"} err="failed to get container status \"c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66\": rpc error: code = NotFound desc = could not find container \"c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66\": container with ID starting with c733cbc8ea60db1779139b1ec1e820d9f662de91179c144adfff270870baaf66 not found: ID does not exist"
Nov 03 17:01:39.385085 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.385042    5605 scope.go:122] "RemoveContainer" containerID="5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c"
Nov 03 17:01:39.385300 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:39.385282    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c\": container with ID starting with 5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c not found: ID does not exist" containerID="5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c"
Nov 03 17:01:39.385337 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.385308    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c"} err="failed to get container status \"5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c\": rpc error: code = NotFound desc = could not find container \"5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c\": container with ID starting with 5cfcd7d44fc30ae0bfad08a3c9a2291e6d275c5c1492419c5a2356dc1df94f4c not found: ID does not exist"
Nov 03 17:01:39.568760 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.568707    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"]
Nov 03 17:01:39.620038 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:39.619953    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-195095-openshift-default-6df8755459-hzm8r"]
Nov 03 17:01:40.308046 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:40.308008    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" path="/var/lib/kubelet/pods/0c75fa68-0957-49ae-a013-0f8fae42dd31/volumes"
Nov 03 17:01:43.677294 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.677271    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 17:01:43.683209 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683190    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/033226c0-d209-48ae-9fef-205e34257ad3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/033226c0-d209-48ae-9fef-205e34257ad3-istiod-ca-cert\") pod \"033226c0-d209-48ae-9fef-205e34257ad3\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") "
Nov 03 17:01:43.683282 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683222    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-kube-api-access-f4f6w\" (UniqueName: \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-kube-api-access-f4f6w\") pod \"033226c0-d209-48ae-9fef-205e34257ad3\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") "
Nov 03 17:01:43.683326 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683314    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-socket\") pod \"033226c0-d209-48ae-9fef-205e34257ad3\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") "
Nov 03 17:01:43.683374 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683340    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/033226c0-d209-48ae-9fef-205e34257ad3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/033226c0-d209-48ae-9fef-205e34257ad3-istio-podinfo\") pod \"033226c0-d209-48ae-9fef-205e34257ad3\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") "
Nov 03 17:01:43.683411 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683378    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-credential-socket\") pod \"033226c0-d209-48ae-9fef-205e34257ad3\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") "
Nov 03 17:01:43.683465 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683411    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-data\") pod \"033226c0-d209-48ae-9fef-205e34257ad3\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") "
Nov 03 17:01:43.683465 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683439    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-envoy\") pod \"033226c0-d209-48ae-9fef-205e34257ad3\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") "
Nov 03 17:01:43.683576 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683474    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-certs\") pod \"033226c0-d209-48ae-9fef-205e34257ad3\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") "
Nov 03 17:01:43.683576 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683531    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-istio-token\" (UniqueName: \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-istio-token\") pod \"033226c0-d209-48ae-9fef-205e34257ad3\" (UID: \"033226c0-d209-48ae-9fef-205e34257ad3\") "
Nov 03 17:01:43.683682 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683566    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/033226c0-d209-48ae-9fef-205e34257ad3-istiod-ca-cert" pod "033226c0-d209-48ae-9fef-205e34257ad3" (UID: "033226c0-d209-48ae-9fef-205e34257ad3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:01:43.683682 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683585    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-socket" pod "033226c0-d209-48ae-9fef-205e34257ad3" (UID: "033226c0-d209-48ae-9fef-205e34257ad3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:43.683816 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683679    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-credential-socket" pod "033226c0-d209-48ae-9fef-205e34257ad3" (UID: "033226c0-d209-48ae-9fef-205e34257ad3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:43.684019 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683878    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-certs" pod "033226c0-d209-48ae-9fef-205e34257ad3" (UID: "033226c0-d209-48ae-9fef-205e34257ad3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:43.684019 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.683941    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-data" pod "033226c0-d209-48ae-9fef-205e34257ad3" (UID: "033226c0-d209-48ae-9fef-205e34257ad3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:43.684019 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.684000    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:43.684019 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.684018    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/033226c0-d209-48ae-9fef-205e34257ad3-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:43.684019 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.684027    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:43.684019 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.684036    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:43.685721 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.685690    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/033226c0-d209-48ae-9fef-205e34257ad3-istio-podinfo" pod "033226c0-d209-48ae-9fef-205e34257ad3" (UID: "033226c0-d209-48ae-9fef-205e34257ad3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 17:01:43.685721 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.685716    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-istio-token" pod "033226c0-d209-48ae-9fef-205e34257ad3" (UID: "033226c0-d209-48ae-9fef-205e34257ad3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:01:43.685917 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.685765    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-envoy" pod "033226c0-d209-48ae-9fef-205e34257ad3" (UID: "033226c0-d209-48ae-9fef-205e34257ad3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:01:43.685917 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.685794    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-kube-api-access-f4f6w" pod "033226c0-d209-48ae-9fef-205e34257ad3" (UID: "033226c0-d209-48ae-9fef-205e34257ad3"). InnerVolumeSpecName "kube-api-access-f4f6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:01:43.785616 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.785529    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4f6w\" (UniqueName: \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-kube-api-access-f4f6w\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:43.785616 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.785555    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/033226c0-d209-48ae-9fef-205e34257ad3-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:43.785616 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.785565    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:43.785616 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.785575    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/033226c0-d209-48ae-9fef-205e34257ad3-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:43.785616 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:43.785584    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/033226c0-d209-48ae-9fef-205e34257ad3-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:44.393167 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.393125    5605 generic.go:356] "Generic (PLEG): container finished" podID="033226c0-d209-48ae-9fef-205e34257ad3" containerID="82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13" exitCode=0
Nov 03 17:01:44.393355 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.393194    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"
Nov 03 17:01:44.393355 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.393194    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" event={"ID":"033226c0-d209-48ae-9fef-205e34257ad3","Type":"ContainerDied","Data":"82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13"}
Nov 03 17:01:44.393355 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.393302    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph" event={"ID":"033226c0-d209-48ae-9fef-205e34257ad3","Type":"ContainerDied","Data":"af3a0ca5e098aa6442e419f72b5cc504dffa3bd1c3bf82025efde3e65d4444e2"}
Nov 03 17:01:44.393355 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.393339    5605 scope.go:122] "RemoveContainer" containerID="82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13"
Nov 03 17:01:44.402501 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.402485    5605 scope.go:122] "RemoveContainer" containerID="df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e"
Nov 03 17:01:44.410588 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.410570    5605 scope.go:122] "RemoveContainer" containerID="82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13"
Nov 03 17:01:44.410863 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:44.410842    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13\": container with ID starting with 82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13 not found: ID does not exist" containerID="82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13"
Nov 03 17:01:44.410931 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.410870    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13"} err="failed to get container status \"82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13\": rpc error: code = NotFound desc = could not find container \"82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13\": container with ID starting with 82585954ed8ae0b29f3cf7d034d1cf08510bef6710a26d731e99e21536a8ce13 not found: ID does not exist"
Nov 03 17:01:44.410931 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.410886    5605 scope.go:122] "RemoveContainer" containerID="df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e"
Nov 03 17:01:44.411227 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:44.411207    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e\": container with ID starting with df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e not found: ID does not exist" containerID="df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e"
Nov 03 17:01:44.411311 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.411234    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e"} err="failed to get container status \"df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e\": rpc error: code = NotFound desc = could not find container \"df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e\": container with ID starting with df740664fb05491c5a62723a64d81b4e6d2a0795a9d6cc9b7986e94c4eefc43e not found: ID does not exist"
Nov 03 17:01:44.565648 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.565608    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"]
Nov 03 17:01:44.616980 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:44.616945    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-963a2f-openshift-default-559b89b79-6xkph"]
Nov 03 17:01:46.308505 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:46.308462    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="033226c0-d209-48ae-9fef-205e34257ad3" path="/var/lib/kubelet/pods/033226c0-d209-48ae-9fef-205e34257ad3/volumes"
Nov 03 17:01:50.086826 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.086794    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"]
Nov 03 17:01:50.087244 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.087064    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd" podUID="1a4e86aa-2c69-4803-994f-99c51afec557" containerName="maas-api" containerID="cri-o://39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a" gracePeriod=30
Nov 03 17:01:50.354045 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.354020    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 17:01:50.425094 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.425061    5605 generic.go:356] "Generic (PLEG): container finished" podID="1a4e86aa-2c69-4803-994f-99c51afec557" containerID="39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a" exitCode=0
Nov 03 17:01:50.425259 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.425103    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"
Nov 03 17:01:50.425259 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.425147    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd" event={"ID":"1a4e86aa-2c69-4803-994f-99c51afec557","Type":"ContainerDied","Data":"39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a"}
Nov 03 17:01:50.425259 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.425196    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd" event={"ID":"1a4e86aa-2c69-4803-994f-99c51afec557","Type":"ContainerDied","Data":"7ba9ae896cf71dfb4d581c450ac02967d12e18160b8eb99a74b12d30c7ee7a12"}
Nov 03 17:01:50.425259 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.425217    5605 scope.go:122] "RemoveContainer" containerID="39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a"
Nov 03 17:01:50.433893 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.433877    5605 scope.go:122] "RemoveContainer" containerID="39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a"
Nov 03 17:01:50.434155 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:01:50.434135    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a\": container with ID starting with 39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a not found: ID does not exist" containerID="39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a"
Nov 03 17:01:50.434227 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.434168    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a"} err="failed to get container status \"39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a\": rpc error: code = NotFound desc = could not find container \"39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a\": container with ID starting with 39feb971c06208912e3ad07d0925f804afef5f83c7d64c01e4c81498e472f50a not found: ID does not exist"
Nov 03 17:01:50.452802 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.452768    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a4e86aa-2c69-4803-994f-99c51afec557-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a4e86aa-2c69-4803-994f-99c51afec557-openshift-service-ca\") pod \"1a4e86aa-2c69-4803-994f-99c51afec557\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") "
Nov 03 17:01:50.452947 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.452877    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a4e86aa-2c69-4803-994f-99c51afec557-kube-api-access-gxn9s\" (UniqueName: \"kubernetes.io/projected/1a4e86aa-2c69-4803-994f-99c51afec557-kube-api-access-gxn9s\") pod \"1a4e86aa-2c69-4803-994f-99c51afec557\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") "
Nov 03 17:01:50.452947 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.452940    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-maas-api-tls\") pod \"1a4e86aa-2c69-4803-994f-99c51afec557\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") "
Nov 03 17:01:50.453076 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.453049    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-metrics-certs\") pod \"1a4e86aa-2c69-4803-994f-99c51afec557\" (UID: \"1a4e86aa-2c69-4803-994f-99c51afec557\") "
Nov 03 17:01:50.453272 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.453244    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a4e86aa-2c69-4803-994f-99c51afec557-openshift-service-ca" pod "1a4e86aa-2c69-4803-994f-99c51afec557" (UID: "1a4e86aa-2c69-4803-994f-99c51afec557"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:01:50.454257 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.454206    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a4e86aa-2c69-4803-994f-99c51afec557-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:50.456069 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.456040    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-maas-api-tls" pod "1a4e86aa-2c69-4803-994f-99c51afec557" (UID: "1a4e86aa-2c69-4803-994f-99c51afec557"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:01:50.456509 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.456470    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4e86aa-2c69-4803-994f-99c51afec557-kube-api-access-gxn9s" pod "1a4e86aa-2c69-4803-994f-99c51afec557" (UID: "1a4e86aa-2c69-4803-994f-99c51afec557"). InnerVolumeSpecName "kube-api-access-gxn9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:01:50.457175 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.457153    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-metrics-certs" pod "1a4e86aa-2c69-4803-994f-99c51afec557" (UID: "1a4e86aa-2c69-4803-994f-99c51afec557"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:01:50.555207 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.555173    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:50.555207 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.555202    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxn9s\" (UniqueName: \"kubernetes.io/projected/1a4e86aa-2c69-4803-994f-99c51afec557-kube-api-access-gxn9s\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:50.555207 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.555212    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4e86aa-2c69-4803-994f-99c51afec557-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:01:50.851529 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.851492    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"]
Nov 03 17:01:50.859261 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:50.859232    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9f2868-745c59db45-hpjzd"]
Nov 03 17:01:52.306984 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:52.306949    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a4e86aa-2c69-4803-994f-99c51afec557" path="/var/lib/kubelet/pods/1a4e86aa-2c69-4803-994f-99c51afec557/volumes"
Nov 03 17:01:54.713123 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:54.713096    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:01:54.731690 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:01:54.731668    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:02:03.392025 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:02:03.391928    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:02:06.174329 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174296    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"]
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174685    5605 cpu_manager.go:414] "containerMap: removing container" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174697    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174710    5605 cpu_manager.go:414] "containerMap: removing container" podUID="1a4e86aa-2c69-4803-994f-99c51afec557" containerName="maas-api"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174715    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a4e86aa-2c69-4803-994f-99c51afec557" containerName="maas-api"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174722    5605 cpu_manager.go:414] "containerMap: removing container" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerName="cleanup"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174749    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerName="cleanup"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174763    5605 cpu_manager.go:414] "containerMap: removing container" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174768    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174775    5605 cpu_manager.go:414] "containerMap: removing container" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerName="cleanup"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174780    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerName="cleanup"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174786    5605 cpu_manager.go:414] "containerMap: removing container" podUID="0233caa0-0726-4bd8-a510-fbc1799b3eab" containerName="payload-pre-processing"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174792    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0233caa0-0726-4bd8-a510-fbc1799b3eab" containerName="payload-pre-processing"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174802    5605 cpu_manager.go:414] "containerMap: removing container" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerName="cleanup"
Nov 03 17:02:06.174794 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174806    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174813    5605 cpu_manager.go:414] "containerMap: removing container" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174819    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174831    5605 cpu_manager.go:414] "containerMap: removing container" podUID="b9df6a16-4e47-4794-a2be-cd38093531bb" containerName="maas-api"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174835    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9df6a16-4e47-4794-a2be-cd38093531bb" containerName="maas-api"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174846    5605 cpu_manager.go:414] "containerMap: removing container" podUID="b058295c-3fa6-409d-83ee-981d4ee60381" containerName="payload-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174851    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b058295c-3fa6-409d-83ee-981d4ee60381" containerName="payload-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174858    5605 cpu_manager.go:414] "containerMap: removing container" podUID="9744816f-937a-43ab-b49d-8cfc4c4522dc" containerName="payload-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174864    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9744816f-937a-43ab-b49d-8cfc4c4522dc" containerName="payload-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174875    5605 cpu_manager.go:414] "containerMap: removing container" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174879    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174887    5605 cpu_manager.go:414] "containerMap: removing container" podUID="c18b4cbe-f908-454c-bf78-612a5b64ae00" containerName="maas-api"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174891    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c18b4cbe-f908-454c-bf78-612a5b64ae00" containerName="maas-api"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174899    5605 cpu_manager.go:414] "containerMap: removing container" podUID="22d32ce6-2fd8-49b4-8ade-ab2c12b24e44" containerName="payload-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174907    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22d32ce6-2fd8-49b4-8ade-ab2c12b24e44" containerName="payload-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174914    5605 cpu_manager.go:414] "containerMap: removing container" podUID="626e2175-2d9f-483c-92bd-f3538bd95f22" containerName="payload-pre-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174920    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="626e2175-2d9f-483c-92bd-f3538bd95f22" containerName="payload-pre-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174928    5605 cpu_manager.go:414] "containerMap: removing container" podUID="97bea05f-9f14-40e0-898a-b957b279858f" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174932    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97bea05f-9f14-40e0-898a-b957b279858f" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174937    5605 cpu_manager.go:414] "containerMap: removing container" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174941    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174952    5605 cpu_manager.go:414] "containerMap: removing container" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174956    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174968    5605 cpu_manager.go:414] "containerMap: removing container" podUID="97bea05f-9f14-40e0-898a-b957b279858f" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.174975    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97bea05f-9f14-40e0-898a-b957b279858f" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175061    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="0233caa0-0726-4bd8-a510-fbc1799b3eab" containerName="payload-pre-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175080    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175087    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175092    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9df6a16-4e47-4794-a2be-cd38093531bb" containerName="maas-api"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175099    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="b058295c-3fa6-409d-83ee-981d4ee60381" containerName="payload-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175105    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="9744816f-937a-43ab-b49d-8cfc4c4522dc" containerName="payload-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175112    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175119    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="c18b4cbe-f908-454c-bf78-612a5b64ae00" containerName="maas-api"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175125    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="22d32ce6-2fd8-49b4-8ade-ab2c12b24e44" containerName="payload-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175132    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175142    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="626e2175-2d9f-483c-92bd-f3538bd95f22" containerName="payload-pre-processing"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175148    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="97bea05f-9f14-40e0-898a-b957b279858f" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175153    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="6376ccbd-aea0-41b3-bd45-cdbe2f999e28" containerName="main"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175159    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175166    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175171    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="97bea05f-9f14-40e0-898a-b957b279858f" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175176    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="97bea05f-9f14-40e0-898a-b957b279858f" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175182    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175187    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a4e86aa-2c69-4803-994f-99c51afec557" containerName="maas-api"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175192    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerName="cleanup"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175196    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="41871919-9219-49c2-abd8-6a38e7e32d2e" containerName="istio-proxy"
Nov 03 17:02:06.175351 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.175203    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerName="cleanup"
Nov 03 17:02:06.179833 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.179816    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.237898 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.237865    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"]
Nov 03 17:02:06.287107 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.287079    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fnmxp\""
Nov 03 17:02:06.287107 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.287091    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 17:02:06.302814 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.302782    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc745e6c-b88qf\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.302973 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.302832    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc5m4\" (UniqueName: \"kubernetes.io/projected/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-kube-api-access-mc5m4\") pod \"maas-api-revoke-sub-cc745e6c-b88qf\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.303046 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.303019    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc745e6c-b88qf\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.404203 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.404169    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc745e6c-b88qf\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.404422 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.404219    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc745e6c-b88qf\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.404422 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.404245    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc5m4\" (UniqueName: \"kubernetes.io/projected/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-kube-api-access-mc5m4\") pod \"maas-api-revoke-sub-cc745e6c-b88qf\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.404975 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.404947    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cc745e6c-b88qf\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.406642 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.406622    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cc745e6c-b88qf\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.464808 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.464739    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc5m4\" (UniqueName: \"kubernetes.io/projected/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-kube-api-access-mc5m4\") pod \"maas-api-revoke-sub-cc745e6c-b88qf\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.492634 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.492601    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:06.869216 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:06.869189    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"]
Nov 03 17:02:06.872552 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:02:06.872525    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6d3e3a_72f6_4a4b_aa2c_dc3ffc4112d7.slice/crio-aaf4c8b2f640e431dc0d00a12343225ab391169bac09a2a607bb1ef8d34def60 WatchSource:0}: Error finding container aaf4c8b2f640e431dc0d00a12343225ab391169bac09a2a607bb1ef8d34def60: Status 404 returned error can't find the container with id aaf4c8b2f640e431dc0d00a12343225ab391169bac09a2a607bb1ef8d34def60
Nov 03 17:02:07.506221 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:07.506120    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf" event={"ID":"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7","Type":"ContainerStarted","Data":"36e95a610dd2388194738b96ffe1b30a045415105c3cf7446e2d8da6504dce41"}
Nov 03 17:02:07.506221 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:07.506183    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf" event={"ID":"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7","Type":"ContainerStarted","Data":"aaf4c8b2f640e431dc0d00a12343225ab391169bac09a2a607bb1ef8d34def60"}
Nov 03 17:02:07.626939 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:07.626882    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf" podStartSLOduration=1.626865244 podStartE2EDuration="1.626865244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:02:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:02:07.626061369 +0000 UTC m=+3013.450512639" watchObservedRunningTime="2026-11-03 17:02:07.626865244 +0000 UTC m=+3013.451316572"
Nov 03 17:02:08.511895 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:08.511857    5605 generic.go:356] "Generic (PLEG): container finished" podID="af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7" containerID="36e95a610dd2388194738b96ffe1b30a045415105c3cf7446e2d8da6504dce41" exitCode=0
Nov 03 17:02:08.512305 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:08.511945    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf" event={"ID":"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7","Type":"ContainerDied","Data":"36e95a610dd2388194738b96ffe1b30a045415105c3cf7446e2d8da6504dce41"}
Nov 03 17:02:09.645929 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:09.645903    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:09.735427 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:09.735398    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-kube-api-access-mc5m4\" (UniqueName: \"kubernetes.io/projected/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-kube-api-access-mc5m4\") pod \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") "
Nov 03 17:02:09.735588 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:09.735482    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-openshift-ca\") pod \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") "
Nov 03 17:02:09.735588 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:09.735573    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-xks-ca\") pod \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\" (UID: \"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7\") "
Nov 03 17:02:09.735872 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:09.735842    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-openshift-ca" pod "af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7" (UID: "af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:02:09.737472 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:09.737452    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-kube-api-access-mc5m4" pod "af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7" (UID: "af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7"). InnerVolumeSpecName "kube-api-access-mc5m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:02:09.737651 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:09.737633    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-xks-ca" pod "af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7" (UID: "af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:02:09.836717 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:09.836692    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-xks-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:09.836717 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:09.836718    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mc5m4\" (UniqueName: \"kubernetes.io/projected/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-kube-api-access-mc5m4\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:09.836916 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:09.836751    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7-maas-api-openshift-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:10.522672 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:10.522638    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf" event={"ID":"af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7","Type":"ContainerDied","Data":"aaf4c8b2f640e431dc0d00a12343225ab391169bac09a2a607bb1ef8d34def60"}
Nov 03 17:02:10.522672 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:10.522661    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"
Nov 03 17:02:10.522916 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:10.522696    5605 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aaf4c8b2f640e431dc0d00a12343225ab391169bac09a2a607bb1ef8d34def60"
Nov 03 17:02:16.455284 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:16.455249    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"]
Nov 03 17:02:16.563815 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:16.563766    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"]
Nov 03 17:02:20.926572 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:20.926530    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"]
Nov 03 17:02:20.927047 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:20.926777    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42" podUID="63ae4568-03c9-444c-82e8-62bbbd007a94" containerName="maas-api" containerID="cri-o://3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9" gracePeriod=30
Nov 03 17:02:20.986132 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:20.986095    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42" podUID="63ae4568-03c9-444c-82e8-62bbbd007a94" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.37:8443/health\": dial tcp 10.133.0.37:8443: connect: connection refused"
Nov 03 17:02:21.178867 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.178812    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 17:02:21.342941 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.342914    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63ae4568-03c9-444c-82e8-62bbbd007a94-kube-api-access-fj6ln\" (UniqueName: \"kubernetes.io/projected/63ae4568-03c9-444c-82e8-62bbbd007a94-kube-api-access-fj6ln\") pod \"63ae4568-03c9-444c-82e8-62bbbd007a94\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") "
Nov 03 17:02:21.343119 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.343021    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63ae4568-03c9-444c-82e8-62bbbd007a94-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ae4568-03c9-444c-82e8-62bbbd007a94-openshift-service-ca\") pod \"63ae4568-03c9-444c-82e8-62bbbd007a94\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") "
Nov 03 17:02:21.343119 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.343075    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-maas-api-tls\") pod \"63ae4568-03c9-444c-82e8-62bbbd007a94\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") "
Nov 03 17:02:21.343119 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.343108    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-metrics-certs\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-metrics-certs\") pod \"63ae4568-03c9-444c-82e8-62bbbd007a94\" (UID: \"63ae4568-03c9-444c-82e8-62bbbd007a94\") "
Nov 03 17:02:21.343398 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.343360    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63ae4568-03c9-444c-82e8-62bbbd007a94-openshift-service-ca" pod "63ae4568-03c9-444c-82e8-62bbbd007a94" (UID: "63ae4568-03c9-444c-82e8-62bbbd007a94"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:02:21.345228 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.345201    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-maas-api-tls" pod "63ae4568-03c9-444c-82e8-62bbbd007a94" (UID: "63ae4568-03c9-444c-82e8-62bbbd007a94"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:02:21.345228 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.345212    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-metrics-certs" pod "63ae4568-03c9-444c-82e8-62bbbd007a94" (UID: "63ae4568-03c9-444c-82e8-62bbbd007a94"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:02:21.345383 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.345260    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63ae4568-03c9-444c-82e8-62bbbd007a94-kube-api-access-fj6ln" pod "63ae4568-03c9-444c-82e8-62bbbd007a94" (UID: "63ae4568-03c9-444c-82e8-62bbbd007a94"). InnerVolumeSpecName "kube-api-access-fj6ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:02:21.444381 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.444287    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:21.444381 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.444319    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63ae4568-03c9-444c-82e8-62bbbd007a94-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:21.444381 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.444329    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj6ln\" (UniqueName: \"kubernetes.io/projected/63ae4568-03c9-444c-82e8-62bbbd007a94-kube-api-access-fj6ln\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:21.444381 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.444338    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ae4568-03c9-444c-82e8-62bbbd007a94-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:21.456032 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.455994    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g" podUID="375c1022-4008-4245-8007-54fede100898" containerName="payload-processing" containerID="cri-o://faa45ac442e7c42390e666c1c5265bf5b7804d37c105cafadf7885bd862b8e47" gracePeriod=25
Nov 03 17:02:21.564480 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.564424    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889" podUID="c4d769bc-b56e-4e04-9d8b-157df8b7e6c2" containerName="payload-pre-processing" containerID="cri-o://abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510" gracePeriod=25
Nov 03 17:02:21.573488 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.573459    5605 generic.go:356] "Generic (PLEG): container finished" podID="63ae4568-03c9-444c-82e8-62bbbd007a94" containerID="3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9" exitCode=0
Nov 03 17:02:21.573626 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.573508    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"
Nov 03 17:02:21.573626 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.573540    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42" event={"ID":"63ae4568-03c9-444c-82e8-62bbbd007a94","Type":"ContainerDied","Data":"3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9"}
Nov 03 17:02:21.573626 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.573595    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42" event={"ID":"63ae4568-03c9-444c-82e8-62bbbd007a94","Type":"ContainerDied","Data":"a334b0721c86d158121fdd9d134d3f8e32085ed06a9485175e7576b95d3b0b73"}
Nov 03 17:02:21.573809 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.573626    5605 scope.go:122] "RemoveContainer" containerID="3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9"
Nov 03 17:02:21.575538 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.575494    5605 generic.go:356] "Generic (PLEG): container finished" podID="375c1022-4008-4245-8007-54fede100898" containerID="faa45ac442e7c42390e666c1c5265bf5b7804d37c105cafadf7885bd862b8e47" exitCode=0
Nov 03 17:02:21.575603 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.575546    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g" event={"ID":"375c1022-4008-4245-8007-54fede100898","Type":"ContainerDied","Data":"faa45ac442e7c42390e666c1c5265bf5b7804d37c105cafadf7885bd862b8e47"}
Nov 03 17:02:21.676684 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.676654    5605 scope.go:122] "RemoveContainer" containerID="3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9"
Nov 03 17:02:21.677085 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:02:21.677058    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9\": container with ID starting with 3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9 not found: ID does not exist" containerID="3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9"
Nov 03 17:02:21.677184 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.677094    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9"} err="failed to get container status \"3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9\": rpc error: code = NotFound desc = could not find container \"3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9\": container with ID starting with 3522f649312b12a67ffabcd6745e0626a63e02aa59240475aa29ef10c91f46a9 not found: ID does not exist"
Nov 03 17:02:21.690701 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.690680    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 17:02:21.746077 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.746048    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"]
Nov 03 17:02:21.800236 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.800206    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42da0f-6c67c657f4-mwl42"]
Nov 03 17:02:21.806244 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.806219    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 17:02:21.849147 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.849118    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/375c1022-4008-4245-8007-54fede100898-kube-api-access-sn4lc\" (UniqueName: \"kubernetes.io/projected/375c1022-4008-4245-8007-54fede100898-kube-api-access-sn4lc\") pod \"375c1022-4008-4245-8007-54fede100898\" (UID: \"375c1022-4008-4245-8007-54fede100898\") "
Nov 03 17:02:21.849328 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.849163    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/375c1022-4008-4245-8007-54fede100898-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/375c1022-4008-4245-8007-54fede100898-plugins-config-volume\") pod \"375c1022-4008-4245-8007-54fede100898\" (UID: \"375c1022-4008-4245-8007-54fede100898\") "
Nov 03 17:02:21.849562 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.849540    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/375c1022-4008-4245-8007-54fede100898-plugins-config-volume" pod "375c1022-4008-4245-8007-54fede100898" (UID: "375c1022-4008-4245-8007-54fede100898"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:02:21.851346 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.851324    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/375c1022-4008-4245-8007-54fede100898-kube-api-access-sn4lc" pod "375c1022-4008-4245-8007-54fede100898" (UID: "375c1022-4008-4245-8007-54fede100898"). InnerVolumeSpecName "kube-api-access-sn4lc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:02:21.950248 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.950222    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-plugins-config-volume\") pod \"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2\" (UID: \"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2\") "
Nov 03 17:02:21.950551 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.950333    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-kube-api-access-wqfk2\" (UniqueName: \"kubernetes.io/projected/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-kube-api-access-wqfk2\") pod \"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2\" (UID: \"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2\") "
Nov 03 17:02:21.950604 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.950568    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn4lc\" (UniqueName: \"kubernetes.io/projected/375c1022-4008-4245-8007-54fede100898-kube-api-access-sn4lc\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:21.950604 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.950588    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/375c1022-4008-4245-8007-54fede100898-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:21.950690 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.950665    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-plugins-config-volume" pod "c4d769bc-b56e-4e04-9d8b-157df8b7e6c2" (UID: "c4d769bc-b56e-4e04-9d8b-157df8b7e6c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:02:21.952545 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:21.952524    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-kube-api-access-wqfk2" pod "c4d769bc-b56e-4e04-9d8b-157df8b7e6c2" (UID: "c4d769bc-b56e-4e04-9d8b-157df8b7e6c2"). InnerVolumeSpecName "kube-api-access-wqfk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:02:22.052551 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.052462    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:22.052551 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.052507    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqfk2\" (UniqueName: \"kubernetes.io/projected/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2-kube-api-access-wqfk2\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:22.307795 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.307693    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63ae4568-03c9-444c-82e8-62bbbd007a94" path="/var/lib/kubelet/pods/63ae4568-03c9-444c-82e8-62bbbd007a94/volumes"
Nov 03 17:02:22.581612 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.581519    5605 generic.go:356] "Generic (PLEG): container finished" podID="c4d769bc-b56e-4e04-9d8b-157df8b7e6c2" containerID="abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510" exitCode=0
Nov 03 17:02:22.581612 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.581560    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"
Nov 03 17:02:22.581612 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.581596    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889" event={"ID":"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2","Type":"ContainerDied","Data":"abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510"}
Nov 03 17:02:22.581896 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.581654    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889" event={"ID":"c4d769bc-b56e-4e04-9d8b-157df8b7e6c2","Type":"ContainerDied","Data":"1a84a959e7b1895642d2dcf5301b18ee32f7d199f628dd8f406b2a9a2ce6280e"}
Nov 03 17:02:22.581896 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.581676    5605 scope.go:122] "RemoveContainer" containerID="abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510"
Nov 03 17:02:22.583886 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.583863    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"
Nov 03 17:02:22.584050 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.583864    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g" event={"ID":"375c1022-4008-4245-8007-54fede100898","Type":"ContainerDied","Data":"2a17cdf050c3e74757ae66367fbdbb99c59db328f6962552dec1d8d8a0ce1064"}
Nov 03 17:02:22.590870 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.590849    5605 scope.go:122] "RemoveContainer" containerID="abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510"
Nov 03 17:02:22.591153 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:02:22.591133    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510\": container with ID starting with abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510 not found: ID does not exist" containerID="abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510"
Nov 03 17:02:22.591218 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.591160    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510"} err="failed to get container status \"abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510\": rpc error: code = NotFound desc = could not find container \"abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510\": container with ID starting with abc873513068e4d021ff724b515e869a4f624ada62e2c972e7128025d118b510 not found: ID does not exist"
Nov 03 17:02:22.591218 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.591176    5605 scope.go:122] "RemoveContainer" containerID="faa45ac442e7c42390e666c1c5265bf5b7804d37c105cafadf7885bd862b8e47"
Nov 03 17:02:22.753064 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.753031    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"]
Nov 03 17:02:22.770461 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.770427    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-509f14-76779775cc-vbb7g"]
Nov 03 17:02:22.886442 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.886409    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"]
Nov 03 17:02:22.893553 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:22.893516    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-509f14-6c999b5d47-6c889"]
Nov 03 17:02:24.314372 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:24.314339    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="375c1022-4008-4245-8007-54fede100898" path="/var/lib/kubelet/pods/375c1022-4008-4245-8007-54fede100898/volumes"
Nov 03 17:02:24.314778 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:24.314691    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4d769bc-b56e-4e04-9d8b-157df8b7e6c2" path="/var/lib/kubelet/pods/c4d769bc-b56e-4e04-9d8b-157df8b7e6c2/volumes"
Nov 03 17:02:33.390865 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:02:33.390826    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:02:39.783481 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:39.783442    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"]
Nov 03 17:02:39.834535 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:39.834498    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"]
Nov 03 17:02:53.260215 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260177    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"]
Nov 03 17:02:53.260751 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260710    5605 cpu_manager.go:414] "containerMap: removing container" podUID="c4d769bc-b56e-4e04-9d8b-157df8b7e6c2" containerName="payload-pre-processing"
Nov 03 17:02:53.260804 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260755    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4d769bc-b56e-4e04-9d8b-157df8b7e6c2" containerName="payload-pre-processing"
Nov 03 17:02:53.260804 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260781    5605 cpu_manager.go:414] "containerMap: removing container" podUID="97bea05f-9f14-40e0-898a-b957b279858f" containerName="cleanup"
Nov 03 17:02:53.260804 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260790    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97bea05f-9f14-40e0-898a-b957b279858f" containerName="cleanup"
Nov 03 17:02:53.260804 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260804    5605 cpu_manager.go:414] "containerMap: removing container" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260811    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c75fa68-0957-49ae-a013-0f8fae42dd31" containerName="istio-proxy"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260830    5605 cpu_manager.go:414] "containerMap: removing container" podUID="375c1022-4008-4245-8007-54fede100898" containerName="payload-processing"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260839    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="375c1022-4008-4245-8007-54fede100898" containerName="payload-processing"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260851    5605 cpu_manager.go:414] "containerMap: removing container" podUID="af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7" containerName="revoke-keys"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260857    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7" containerName="revoke-keys"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260870    5605 cpu_manager.go:414] "containerMap: removing container" podUID="63ae4568-03c9-444c-82e8-62bbbd007a94" containerName="maas-api"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260877    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63ae4568-03c9-444c-82e8-62bbbd007a94" containerName="maas-api"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260890    5605 cpu_manager.go:414] "containerMap: removing container" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerName="cleanup"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260897    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerName="cleanup"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260907    5605 cpu_manager.go:414] "containerMap: removing container" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerName="cleanup"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260914    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerName="cleanup"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260939    5605 cpu_manager.go:414] "containerMap: removing container" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy"
Nov 03 17:02:53.260949 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.260946    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="033226c0-d209-48ae-9fef-205e34257ad3" containerName="istio-proxy"
Nov 03 17:02:53.261493 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.261045    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4d769bc-b56e-4e04-9d8b-157df8b7e6c2" containerName="payload-pre-processing"
Nov 03 17:02:53.261493 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.261059    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="375c1022-4008-4245-8007-54fede100898" containerName="payload-processing"
Nov 03 17:02:53.261493 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.261069    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7" containerName="revoke-keys"
Nov 03 17:02:53.261493 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.261078    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="63ae4568-03c9-444c-82e8-62bbbd007a94" containerName="maas-api"
Nov 03 17:02:53.261493 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.261088    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="30f6c764-7a1b-48e0-9b28-ddeb3b151f06" containerName="cleanup"
Nov 03 17:02:53.261493 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.261097    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="b378df9b-dd3b-40c3-8886-1b67696431b3" containerName="cleanup"
Nov 03 17:02:53.264473 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.264454    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.333774 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.333740    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"]
Nov 03 17:02:53.334795 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.334766    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-77523744-6nhwn\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.334937 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.334836    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-77523744-6nhwn\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.335008 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.334985    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kw4s\" (UniqueName: \"kubernetes.io/projected/1bbe7d4a-0746-408b-935e-7577403b3a2d-kube-api-access-4kw4s\") pod \"maas-api-revoke-sub-77523744-6nhwn\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.356105 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.356083    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 17:02:53.370892 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.370868    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fnmxp\""
Nov 03 17:02:53.436437 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.436392    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kw4s\" (UniqueName: \"kubernetes.io/projected/1bbe7d4a-0746-408b-935e-7577403b3a2d-kube-api-access-4kw4s\") pod \"maas-api-revoke-sub-77523744-6nhwn\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.436618 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.436462    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-77523744-6nhwn\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.436618 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.436494    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-77523744-6nhwn\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.437266 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.437246    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-77523744-6nhwn\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.438996 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.438976    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-77523744-6nhwn\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.499606 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.499571    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kw4s\" (UniqueName: \"kubernetes.io/projected/1bbe7d4a-0746-408b-935e-7577403b3a2d-kube-api-access-4kw4s\") pod \"maas-api-revoke-sub-77523744-6nhwn\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.577808 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.577697    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:53.952558 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:53.952530    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"]
Nov 03 17:02:53.955300 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:02:53.955265    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bbe7d4a_0746_408b_935e_7577403b3a2d.slice/crio-937eb11cab260bf81de0ad2814d94526932a0b3e48a42928007f506560c7c67f WatchSource:0}: Error finding container 937eb11cab260bf81de0ad2814d94526932a0b3e48a42928007f506560c7c67f: Status 404 returned error can't find the container with id 937eb11cab260bf81de0ad2814d94526932a0b3e48a42928007f506560c7c67f
Nov 03 17:02:54.742233 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:54.742187    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn" event={"ID":"1bbe7d4a-0746-408b-935e-7577403b3a2d","Type":"ContainerStarted","Data":"bae8a6e3fe7b6bdabd4a2b429d600cb7fee722a5542e7e2a8989fdae967a9a1b"}
Nov 03 17:02:54.742641 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:54.742256    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn" event={"ID":"1bbe7d4a-0746-408b-935e-7577403b3a2d","Type":"ContainerStarted","Data":"937eb11cab260bf81de0ad2814d94526932a0b3e48a42928007f506560c7c67f"}
Nov 03 17:02:54.803458 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:54.803410    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main" containerID="cri-o://d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8" gracePeriod=45
Nov 03 17:02:54.853661 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:54.853624    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main" containerID="cri-o://85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180" gracePeriod=45
Nov 03 17:02:54.867228 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:54.867181    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn" podStartSLOduration=1.867166723 podStartE2EDuration="1.867166723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:02:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:02:54.866264935 +0000 UTC m=+3060.690716203" watchObservedRunningTime="2026-11-03 17:02:54.867166723 +0000 UTC m=+3060.691618031"
Nov 03 17:02:55.112208 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.112185    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 17:02:55.115093 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.115060    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 17:02:55.252752 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.252706    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-home\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-home\") pod \"e65c45be-2a14-4d0f-b618-69d01cb6926c\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") "
Nov 03 17:02:55.252938 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.252767    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-dshm\") pod \"e65c45be-2a14-4d0f-b618-69d01cb6926c\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") "
Nov 03 17:02:55.252938 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.252806    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-model-cache\") pod \"e65c45be-2a14-4d0f-b618-69d01cb6926c\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") "
Nov 03 17:02:55.252938 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.252827    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-kube-api-access-5sdpk\" (UniqueName: \"kubernetes.io/projected/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-kube-api-access-5sdpk\") pod \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") "
Nov 03 17:02:55.252938 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.252853    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tls-certs\" (UniqueName: \"kubernetes.io/secret/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tls-certs\") pod \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") "
Nov 03 17:02:55.252938 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.252882    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e65c45be-2a14-4d0f-b618-69d01cb6926c-tls-certs\" (UniqueName: \"kubernetes.io/secret/e65c45be-2a14-4d0f-b618-69d01cb6926c-tls-certs\") pod \"e65c45be-2a14-4d0f-b618-69d01cb6926c\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") "
Nov 03 17:02:55.252938 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.252929    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-tmp-dir\") pod \"e65c45be-2a14-4d0f-b618-69d01cb6926c\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") "
Nov 03 17:02:55.253206 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.252962    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-home\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-home\") pod \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") "
Nov 03 17:02:55.253206 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253043    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tmp-dir\") pod \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") "
Nov 03 17:02:55.253206 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253054    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-home" pod "e65c45be-2a14-4d0f-b618-69d01cb6926c" (UID: "e65c45be-2a14-4d0f-b618-69d01cb6926c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:02:55.253206 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253098    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-model-cache\") pod \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") "
Nov 03 17:02:55.253206 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253134    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-dshm\") pod \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\" (UID: \"bde4efc7-f40e-4467-b076-1e2d6c47a0b0\") "
Nov 03 17:02:55.253206 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253184    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e65c45be-2a14-4d0f-b618-69d01cb6926c-kube-api-access-bvrbp\" (UniqueName: \"kubernetes.io/projected/e65c45be-2a14-4d0f-b618-69d01cb6926c-kube-api-access-bvrbp\") pod \"e65c45be-2a14-4d0f-b618-69d01cb6926c\" (UID: \"e65c45be-2a14-4d0f-b618-69d01cb6926c\") "
Nov 03 17:02:55.253537 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253261    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-tmp-dir" pod "e65c45be-2a14-4d0f-b618-69d01cb6926c" (UID: "e65c45be-2a14-4d0f-b618-69d01cb6926c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:02:55.253537 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253436    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tmp-dir" pod "bde4efc7-f40e-4467-b076-1e2d6c47a0b0" (UID: "bde4efc7-f40e-4467-b076-1e2d6c47a0b0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:02:55.253537 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253448    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-home" pod "bde4efc7-f40e-4467-b076-1e2d6c47a0b0" (UID: "bde4efc7-f40e-4467-b076-1e2d6c47a0b0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:02:55.253806 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253630    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-model-cache" pod "bde4efc7-f40e-4467-b076-1e2d6c47a0b0" (UID: "bde4efc7-f40e-4467-b076-1e2d6c47a0b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:02:55.253806 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253789    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.254400 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253918    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.254400 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253939    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.254400 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.253952    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.254400 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.254062    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-model-cache" pod "e65c45be-2a14-4d0f-b618-69d01cb6926c" (UID: "e65c45be-2a14-4d0f-b618-69d01cb6926c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:02:55.256341 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.256312    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tls-certs" pod "bde4efc7-f40e-4467-b076-1e2d6c47a0b0" (UID: "bde4efc7-f40e-4467-b076-1e2d6c47a0b0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:02:55.256462 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.256339    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e65c45be-2a14-4d0f-b618-69d01cb6926c-kube-api-access-bvrbp" pod "e65c45be-2a14-4d0f-b618-69d01cb6926c" (UID: "e65c45be-2a14-4d0f-b618-69d01cb6926c"). InnerVolumeSpecName "kube-api-access-bvrbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:02:55.256545 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.256526    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e65c45be-2a14-4d0f-b618-69d01cb6926c-tls-certs" pod "e65c45be-2a14-4d0f-b618-69d01cb6926c" (UID: "e65c45be-2a14-4d0f-b618-69d01cb6926c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:02:55.256857 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.256833    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-dshm" pod "e65c45be-2a14-4d0f-b618-69d01cb6926c" (UID: "e65c45be-2a14-4d0f-b618-69d01cb6926c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:02:55.256963 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.256865    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-kube-api-access-5sdpk" pod "bde4efc7-f40e-4467-b076-1e2d6c47a0b0" (UID: "bde4efc7-f40e-4467-b076-1e2d6c47a0b0"). InnerVolumeSpecName "kube-api-access-5sdpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:02:55.256963 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.256934    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-dshm" pod "bde4efc7-f40e-4467-b076-1e2d6c47a0b0" (UID: "bde4efc7-f40e-4467-b076-1e2d6c47a0b0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:02:55.354390 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.354352    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e65c45be-2a14-4d0f-b618-69d01cb6926c-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.354390 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.354384    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.354390 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.354397    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.354629 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.354411    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvrbp\" (UniqueName: \"kubernetes.io/projected/e65c45be-2a14-4d0f-b618-69d01cb6926c-kube-api-access-bvrbp\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.354629 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.354424    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.354629 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.354436    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e65c45be-2a14-4d0f-b618-69d01cb6926c-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.354629 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.354446    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sdpk\" (UniqueName: \"kubernetes.io/projected/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-kube-api-access-5sdpk\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.354629 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.354458    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bde4efc7-f40e-4467-b076-1e2d6c47a0b0-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:55.747242 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.747145    5605 generic.go:356] "Generic (PLEG): container finished" podID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerID="d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8" exitCode=0
Nov 03 17:02:55.747242 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.747193    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"
Nov 03 17:02:55.747242 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.747220    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" event={"ID":"bde4efc7-f40e-4467-b076-1e2d6c47a0b0","Type":"ContainerDied","Data":"d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8"}
Nov 03 17:02:55.747870 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.747253    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd" event={"ID":"bde4efc7-f40e-4467-b076-1e2d6c47a0b0","Type":"ContainerDied","Data":"511270f20b48f7a69dabb4db0befb24b199342ef477842a753a285809a5c4430"}
Nov 03 17:02:55.747870 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.747272    5605 scope.go:122] "RemoveContainer" containerID="d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8"
Nov 03 17:02:55.748605 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.748581    5605 generic.go:356] "Generic (PLEG): container finished" podID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerID="85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180" exitCode=0
Nov 03 17:02:55.748744 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.748615    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"
Nov 03 17:02:55.748744 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.748674    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" event={"ID":"e65c45be-2a14-4d0f-b618-69d01cb6926c","Type":"ContainerDied","Data":"85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180"}
Nov 03 17:02:55.748744 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.748721    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f" event={"ID":"e65c45be-2a14-4d0f-b618-69d01cb6926c","Type":"ContainerDied","Data":"55779d38190192a263da552f79bb152257a364ec10225bdd2230f60655ce92da"}
Nov 03 17:02:55.750479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.750448    5605 generic.go:356] "Generic (PLEG): container finished" podID="1bbe7d4a-0746-408b-935e-7577403b3a2d" containerID="bae8a6e3fe7b6bdabd4a2b429d600cb7fee722a5542e7e2a8989fdae967a9a1b" exitCode=0
Nov 03 17:02:55.750586 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.750517    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn" event={"ID":"1bbe7d4a-0746-408b-935e-7577403b3a2d","Type":"ContainerDied","Data":"bae8a6e3fe7b6bdabd4a2b429d600cb7fee722a5542e7e2a8989fdae967a9a1b"}
Nov 03 17:02:55.757121 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.757103    5605 scope.go:122] "RemoveContainer" containerID="d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8"
Nov 03 17:02:55.757449 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:02:55.757429    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8\": container with ID starting with d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8 not found: ID does not exist" containerID="d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8"
Nov 03 17:02:55.757511 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.757457    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8"} err="failed to get container status \"d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8\": rpc error: code = NotFound desc = could not find container \"d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8\": container with ID starting with d0000b4b68df9cab8939c8aac758358b875654af48fe0e8130a8fb005e66bbd8 not found: ID does not exist"
Nov 03 17:02:55.757511 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.757477    5605 scope.go:122] "RemoveContainer" containerID="85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180"
Nov 03 17:02:55.765464 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.765447    5605 scope.go:122] "RemoveContainer" containerID="85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180"
Nov 03 17:02:55.765753 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:02:55.765711    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180\": container with ID starting with 85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180 not found: ID does not exist" containerID="85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180"
Nov 03 17:02:55.765867 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.765803    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180"} err="failed to get container status \"85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180\": rpc error: code = NotFound desc = could not find container \"85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180\": container with ID starting with 85a3b43717fb520516a4e5523a7c8a6b8ffb9bb00702a10cdda68f4ac70df180 not found: ID does not exist"
Nov 03 17:02:55.920232 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.920196    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"]
Nov 03 17:02:55.971567 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:55.971532    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/distinct-w2-c88d54-kserve-6db69d99f7-xkkjd"]
Nov 03 17:02:56.086785 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.086755    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"]
Nov 03 17:02:56.092993 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.092965    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-c88d54/simulator-w2-c88d54-kserve-75899c4c6c-whx6f"]
Nov 03 17:02:56.307658 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.307628    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" path="/var/lib/kubelet/pods/bde4efc7-f40e-4467-b076-1e2d6c47a0b0/volumes"
Nov 03 17:02:56.308076 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.308062    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" path="/var/lib/kubelet/pods/e65c45be-2a14-4d0f-b618-69d01cb6926c/volumes"
Nov 03 17:02:56.884099 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.884072    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:56.966614 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.966585    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-xks-ca\") pod \"1bbe7d4a-0746-408b-935e-7577403b3a2d\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") "
Nov 03 17:02:56.966791 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.966680    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bbe7d4a-0746-408b-935e-7577403b3a2d-kube-api-access-4kw4s\" (UniqueName: \"kubernetes.io/projected/1bbe7d4a-0746-408b-935e-7577403b3a2d-kube-api-access-4kw4s\") pod \"1bbe7d4a-0746-408b-935e-7577403b3a2d\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") "
Nov 03 17:02:56.966791 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.966762    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-openshift-ca\") pod \"1bbe7d4a-0746-408b-935e-7577403b3a2d\" (UID: \"1bbe7d4a-0746-408b-935e-7577403b3a2d\") "
Nov 03 17:02:56.967049 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.967030    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-openshift-ca" pod "1bbe7d4a-0746-408b-935e-7577403b3a2d" (UID: "1bbe7d4a-0746-408b-935e-7577403b3a2d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:02:56.968740 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.968705    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bbe7d4a-0746-408b-935e-7577403b3a2d-kube-api-access-4kw4s" pod "1bbe7d4a-0746-408b-935e-7577403b3a2d" (UID: "1bbe7d4a-0746-408b-935e-7577403b3a2d"). InnerVolumeSpecName "kube-api-access-4kw4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:02:56.968905 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:56.968887    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-xks-ca" pod "1bbe7d4a-0746-408b-935e-7577403b3a2d" (UID: "1bbe7d4a-0746-408b-935e-7577403b3a2d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:02:57.068088 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:57.067994    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-openshift-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:57.068088 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:57.068029    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1bbe7d4a-0746-408b-935e-7577403b3a2d-maas-api-xks-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:57.068088 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:57.068045    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kw4s\" (UniqueName: \"kubernetes.io/projected/1bbe7d4a-0746-408b-935e-7577403b3a2d-kube-api-access-4kw4s\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:02:57.760144 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:57.760112    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"
Nov 03 17:02:57.760144 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:57.760131    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn" event={"ID":"1bbe7d4a-0746-408b-935e-7577403b3a2d","Type":"ContainerDied","Data":"937eb11cab260bf81de0ad2814d94526932a0b3e48a42928007f506560c7c67f"}
Nov 03 17:02:57.760366 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:02:57.760202    5605 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="937eb11cab260bf81de0ad2814d94526932a0b3e48a42928007f506560c7c67f"
Nov 03 17:03:03.392226 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:03:03.392187    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:03:33.390479 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:03:33.390368    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:04:03.393258 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:04:03.393219    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:04:33.391245 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:04:33.391210    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:05:03.392030 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:05:03.391942    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:05:21.367123 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.367086    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"]
Nov 03 17:05:21.367615 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.367413    5605 cpu_manager.go:414] "containerMap: removing container" podUID="1bbe7d4a-0746-408b-935e-7577403b3a2d" containerName="revoke-keys"
Nov 03 17:05:21.367615 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.367425    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1bbe7d4a-0746-408b-935e-7577403b3a2d" containerName="revoke-keys"
Nov 03 17:05:21.367615 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.367452    5605 cpu_manager.go:414] "containerMap: removing container" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main"
Nov 03 17:05:21.367615 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.367459    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main"
Nov 03 17:05:21.367615 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.367465    5605 cpu_manager.go:414] "containerMap: removing container" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main"
Nov 03 17:05:21.367615 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.367470    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main"
Nov 03 17:05:21.367615 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.367521    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="bde4efc7-f40e-4467-b076-1e2d6c47a0b0" containerName="main"
Nov 03 17:05:21.367615 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.367529    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="e65c45be-2a14-4d0f-b618-69d01cb6926c" containerName="main"
Nov 03 17:05:21.367615 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.367537    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="1bbe7d4a-0746-408b-935e-7577403b3a2d" containerName="revoke-keys"
Nov 03 17:05:21.370565 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.370542    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.456255 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.456222    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.456433 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.456321    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.456433 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.456408    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.456576 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.456474    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b8d456-e273-4b57-94b7-7bffc3831f87-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.456576 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.456506    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.456712 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.456579    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km7pd\" (UniqueName: \"kubernetes.io/projected/04b8d456-e273-4b57-94b7-7bffc3831f87-kube-api-access-km7pd\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.456712 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.456605    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.456712 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.456625    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.456712 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.456703    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.477807 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.477775    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"]
Nov 03 17:05:21.557395 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557351    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km7pd\" (UniqueName: \"kubernetes.io/projected/04b8d456-e273-4b57-94b7-7bffc3831f87-kube-api-access-km7pd\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.557395 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557400    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.557660 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557430    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.557660 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557465    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.557660 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557503    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.557660 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557524    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.557660 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557559    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.557660 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557626    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b8d456-e273-4b57-94b7-7bffc3831f87-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.557660 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557656    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.558112 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557930    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.558112 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.557944    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.558215 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.558183    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.558282 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.558194    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.558503 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.558298    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b8d456-e273-4b57-94b7-7bffc3831f87-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.560212 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.560188    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.560323 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.560256    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.617633 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.617567    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b8d456-e273-4b57-94b7-7bffc3831f87-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.617888 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.617865    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-km7pd\" (UniqueName: \"kubernetes.io/projected/04b8d456-e273-4b57-94b7-7bffc3831f87-kube-api-access-km7pd\") pod \"maas-default-gateway-openshift-default-5bc884dd87-zgbzx\" (UID: \"04b8d456-e273-4b57-94b7-7bffc3831f87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:21.683525 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:21.683492    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:22.114777 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:22.114750    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"]
Nov 03 17:05:22.117173 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:05:22.117145    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b8d456_e273_4b57_94b7_7bffc3831f87.slice/crio-2f3ffa556f7e8702e00247cfdf2f87229c9005263792b377f46c6ac8784d6c0e WatchSource:0}: Error finding container 2f3ffa556f7e8702e00247cfdf2f87229c9005263792b377f46c6ac8784d6c0e: Status 404 returned error can't find the container with id 2f3ffa556f7e8702e00247cfdf2f87229c9005263792b377f46c6ac8784d6c0e
Nov 03 17:05:22.119208 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:22.119176    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:05:22.119297 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:22.119241    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:05:22.119297 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:22.119269    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:05:22.383157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:22.383051    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" event={"ID":"04b8d456-e273-4b57-94b7-7bffc3831f87","Type":"ContainerStarted","Data":"d6d045b2ff97a3d9efd3c872a53968470fddd6fe078cde613249f895ba530712"}
Nov 03 17:05:22.383157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:22.383138    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" event={"ID":"04b8d456-e273-4b57-94b7-7bffc3831f87","Type":"ContainerStarted","Data":"2f3ffa556f7e8702e00247cfdf2f87229c9005263792b377f46c6ac8784d6c0e"}
Nov 03 17:05:22.508311 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:22.508245    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podStartSLOduration=1.508223968 podStartE2EDuration="1.508223968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:05:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:05:22.507623345 +0000 UTC m=+3208.332074617" watchObservedRunningTime="2026-11-03 17:05:22.508223968 +0000 UTC m=+3208.332675241"
Nov 03 17:05:22.692984 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:22.692830    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:23.705669 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:23.705631    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-zgbzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Nov 03 17:05:23.706155 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:23.705845    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podUID="04b8d456-e273-4b57-94b7-7bffc3831f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Nov 03 17:05:24.697766 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:24.695243    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-zgbzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Nov 03 17:05:24.697766 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:24.695313    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podUID="04b8d456-e273-4b57-94b7-7bffc3831f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Nov 03 17:05:25.700753 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:25.695866    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-zgbzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Nov 03 17:05:25.700753 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:25.695927    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podUID="04b8d456-e273-4b57-94b7-7bffc3831f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Nov 03 17:05:26.701754 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:26.695181    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-zgbzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Nov 03 17:05:26.701754 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:26.695289    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podUID="04b8d456-e273-4b57-94b7-7bffc3831f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Nov 03 17:05:27.694163 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:27.694114    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-zgbzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Nov 03 17:05:27.694469 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:27.694437    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podUID="04b8d456-e273-4b57-94b7-7bffc3831f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Nov 03 17:05:28.684131 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:28.684096    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-zgbzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Nov 03 17:05:28.684672 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:28.684149    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podUID="04b8d456-e273-4b57-94b7-7bffc3831f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Nov 03 17:05:30.308288 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:30.308263    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:30.308654 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:30.308519    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:30.309346 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:30.309330    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx"
Nov 03 17:05:33.390951 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:05:33.390911    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:05:36.494018 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.493980    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"]
Nov 03 17:05:36.498239 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.498214    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.635755 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.633064    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"]
Nov 03 17:05:36.635755 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.634436    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.635755 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.634487    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.635755 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.634593    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.635755 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.634633    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdfw8\" (UniqueName: \"kubernetes.io/projected/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-kube-api-access-gdfw8\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.635755 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.634693    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.635755 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.634782    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.635755 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.634827    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.635755 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.634859    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.635755 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.634916    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.736262 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.736213    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.736479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.736311    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.736479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.736341    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.736479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.736372    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.736479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.736414    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.736479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.736453    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.736822 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.736491    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.736822 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.736574    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.736822 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.736609    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdfw8\" (UniqueName: \"kubernetes.io/projected/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-kube-api-access-gdfw8\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.737559 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.737532    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.737884 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.737858    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.738158 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.738135    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.738414 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.738392    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.739430 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.739401    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.742446 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.742398    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.748483 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.748420    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.801260 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.801200    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdfw8\" (UniqueName: \"kubernetes.io/projected/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-kube-api-access-gdfw8\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:36.837134 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:36.835415    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-gjzpm\" (UID: \"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:37.117486 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:37.117447    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:37.585709 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:37.585678    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"]
Nov 03 17:05:37.625133 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:05:37.625091    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd9078ae_d2e0_4c93_8e6b_07d8f19dd7da.slice/crio-a4465310c3b874f4b27047eb1469265dc5300e9c9c4b288efd63ed30c64bf95f WatchSource:0}: Error finding container a4465310c3b874f4b27047eb1469265dc5300e9c9c4b288efd63ed30c64bf95f: Status 404 returned error can't find the container with id a4465310c3b874f4b27047eb1469265dc5300e9c9c4b288efd63ed30c64bf95f
Nov 03 17:05:37.628354 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:37.628308    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:05:37.628466 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:37.628405    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:05:37.628552 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:37.628484    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:05:38.476780 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:38.476743    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" event={"ID":"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da","Type":"ContainerStarted","Data":"cf7b494d3ded1a1859ac4b4afe3bc8363caa55edd47a8fad3ac1f49ecc051b78"}
Nov 03 17:05:38.477062 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:38.476795    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" event={"ID":"dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da","Type":"ContainerStarted","Data":"a4465310c3b874f4b27047eb1469265dc5300e9c9c4b288efd63ed30c64bf95f"}
Nov 03 17:05:38.708346 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:38.708278    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podStartSLOduration=2.708261299 podStartE2EDuration="2.708261299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:05:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:05:38.656275633 +0000 UTC m=+3224.480726923" watchObservedRunningTime="2026-11-03 17:05:38.708261299 +0000 UTC m=+3224.532712570"
Nov 03 17:05:39.118037 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:39.118001    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:39.119312 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:39.119289    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Nov 03 17:05:39.119390 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:39.119336    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Nov 03 17:05:40.118901 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:40.118858    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Nov 03 17:05:40.119420 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:40.118926    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Nov 03 17:05:41.118477 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:41.118439    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Nov 03 17:05:41.118657 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:41.118491    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Nov 03 17:05:42.118288 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:42.118249    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Nov 03 17:05:42.118678 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:42.118308    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Nov 03 17:05:43.118583 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:43.118546    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Nov 03 17:05:43.118977 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:43.118599    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Nov 03 17:05:44.118697 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:44.118658    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Nov 03 17:05:44.119158 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:44.118711    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Nov 03 17:05:45.119186 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:45.119151    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Nov 03 17:05:45.119587 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:45.119208    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Nov 03 17:05:46.122841 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:46.122805    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:46.123347 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:46.123163    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:05:46.124036 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:05:46.124016    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm"
Nov 03 17:06:03.390936 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:06:03.390894    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:06:06.755993 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.755960    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"]
Nov 03 17:06:06.759618 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.759602    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.838232 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.838199    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.838232 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.838236    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.838443 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.838337    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fblrg\" (UniqueName: \"kubernetes.io/projected/5c54230f-c0e9-4672-b919-e6bd3383c411-kube-api-access-fblrg\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.838443 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.838394    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.838517 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.838475    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.838517 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.838492    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.838589 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.838516    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.838589 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.838561    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c54230f-c0e9-4672-b919-e6bd3383c411-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.838653 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.838636    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.866978 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.866949    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"]
Nov 03 17:06:06.939143 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939107    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fblrg\" (UniqueName: \"kubernetes.io/projected/5c54230f-c0e9-4672-b919-e6bd3383c411-kube-api-access-fblrg\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939321 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939161    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939321 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939210    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939321 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939227    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939321 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939262    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939494 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939403    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c54230f-c0e9-4672-b919-e6bd3383c411-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939494 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939456    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939494 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939483    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939648 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939504    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939713 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939687    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939900 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939864    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.939992 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.939873    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.940317 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.940293    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.940447 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.940424    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c54230f-c0e9-4672-b919-e6bd3383c411-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.942091 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.942062    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:06.942191 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:06.942174    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:07.000106 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:07.000065    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fblrg\" (UniqueName: \"kubernetes.io/projected/5c54230f-c0e9-4672-b919-e6bd3383c411-kube-api-access-fblrg\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:07.000394 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:07.000374    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c54230f-c0e9-4672-b919-e6bd3383c411-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-rj7rb\" (UID: \"5c54230f-c0e9-4672-b919-e6bd3383c411\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:07.072269 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:07.072181    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:07.507467 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:07.507440    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"]
Nov 03 17:06:07.509419 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:06:07.509389    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c54230f_c0e9_4672_b919_e6bd3383c411.slice/crio-8e5d324655d65f6bb91be43c5760d0e20809bbfa7e7432c719ff6dcbaf6081f2 WatchSource:0}: Error finding container 8e5d324655d65f6bb91be43c5760d0e20809bbfa7e7432c719ff6dcbaf6081f2: Status 404 returned error can't find the container with id 8e5d324655d65f6bb91be43c5760d0e20809bbfa7e7432c719ff6dcbaf6081f2
Nov 03 17:06:07.511540 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:07.511506    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:06:07.511647 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:07.511586    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:06:07.511647 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:07.511629    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:06:07.648626 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:07.648527    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" event={"ID":"5c54230f-c0e9-4672-b919-e6bd3383c411","Type":"ContainerStarted","Data":"71529ee6cda52c61d648943895e1da2b92d32db48e562289f90ddc44161845fc"}
Nov 03 17:06:07.648626 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:07.648593    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" event={"ID":"5c54230f-c0e9-4672-b919-e6bd3383c411","Type":"ContainerStarted","Data":"8e5d324655d65f6bb91be43c5760d0e20809bbfa7e7432c719ff6dcbaf6081f2"}
Nov 03 17:06:07.771999 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:07.771872    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podStartSLOduration=1.771853268 podStartE2EDuration="1.771853268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:06:07.771111711 +0000 UTC m=+3253.595562983" watchObservedRunningTime="2026-11-03 17:06:07.771853268 +0000 UTC m=+3253.596304541"
Nov 03 17:06:08.072443 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:08.072353    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:08.074080 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:08.074058    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Nov 03 17:06:08.074136 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:08.074109    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Nov 03 17:06:09.073578 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:09.073537    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Nov 03 17:06:09.074134 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:09.073624    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Nov 03 17:06:10.072791 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:10.072750    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Nov 03 17:06:10.072975 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:10.072811    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Nov 03 17:06:11.073282 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:11.073241    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Nov 03 17:06:11.073829 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:11.073308    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Nov 03 17:06:12.072822 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:12.072781    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Nov 03 17:06:12.073045 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:12.072845    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Nov 03 17:06:13.073227 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:13.073174    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Nov 03 17:06:13.073806 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:13.073263    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Nov 03 17:06:14.072620 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:14.072573    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Nov 03 17:06:14.072825 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:14.072641    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Nov 03 17:06:15.072713 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:15.072674    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Nov 03 17:06:15.073126 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:15.072763    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Nov 03 17:06:17.073372 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:17.073326    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:06:17.073924 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:17.073414    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:06:18.114933 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:18.114877    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:06:18.115468 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:18.114940    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:06:18.509530 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:18.509433    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:18.510021 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:18.509989    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:18.510691 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:18.510675    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb"
Nov 03 17:06:33.391463 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:06:33.391372    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:06:54.742497 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:54.742470    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:06:54.760390 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:06:54.760369    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:07:03.390967 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:07:03.390923    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:07:10.063355 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:07:10.063314    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"]
Nov 03 17:07:10.114284 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:07:10.114252    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cc745e6c-b88qf"]
Nov 03 17:07:10.307314 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:07:10.307281    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7" path="/var/lib/kubelet/pods/af6d3e3a-72f6-4a4b-aa2c-dc3ffc4112d7/volumes"
Nov 03 17:07:33.390074 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:07:33.390028    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:07:47.166982 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:07:47.166942    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:07:47.167687 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:07:47.167653    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:07:57.066888 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:07:57.066849    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"]
Nov 03 17:07:57.118074 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:07:57.118033    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-77523744-6nhwn"]
Nov 03 17:07:58.307214 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:07:58.307182    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1bbe7d4a-0746-408b-935e-7577403b3a2d" path="/var/lib/kubelet/pods/1bbe7d4a-0746-408b-935e-7577403b3a2d/volumes"
Nov 03 17:08:03.392296 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:08:03.392259    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:08:17.171424 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:08:17.170879    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:08:17.171424 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:08:17.170954    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:08:33.391507 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:08:33.391471    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:08:55.879044 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:08:55.879014    5605 scope.go:122] "RemoveContainer" containerID="bae8a6e3fe7b6bdabd4a2b429d600cb7fee722a5542e7e2a8989fdae967a9a1b"
Nov 03 17:08:55.887554 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:08:55.887537    5605 scope.go:122] "RemoveContainer" containerID="36e95a610dd2388194738b96ffe1b30a045415105c3cf7446e2d8da6504dce41"
Nov 03 17:09:03.542756 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:09:03.539461    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:09:04.584892 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:09:04.584857    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:09:04.585645 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:09:04.585612    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:09:33.392014 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:09:33.391899    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:10:03.394933 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:10:03.394894    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:10:33.391438 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:10:33.391400    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:11:03.390772 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:11:03.390664    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:11:33.391685 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:11:33.391646    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:11:34.084063 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.084020    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"]
Nov 03 17:11:34.088145 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.088116    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.146758 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.146697    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-433fd6-openshift-default-dockercfg-grjcj\""
Nov 03 17:11:34.156276 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.156246    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-certs\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.156427 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.156294    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-istio-token\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.156427 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.156415    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-data\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.156533 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.156463    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/226f0858-e297-4699-9d2b-59144a762bf1-istio-podinfo\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.156533 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.156487    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-credential-socket\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.156533 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.156509    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-socket\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.156533 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.156529    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-envoy\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.156690 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.156637    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl2zf\" (UniqueName: \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-kube-api-access-hl2zf\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.156690 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.156680    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/226f0858-e297-4699-9d2b-59144a762bf1-istiod-ca-cert\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.226435 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.226395    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"]
Nov 03 17:11:34.257744 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.257697    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/226f0858-e297-4699-9d2b-59144a762bf1-istio-podinfo\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.257939 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.257759    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-credential-socket\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.257939 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.257804    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-socket\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.257939 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.257838    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-envoy\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.258128 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.257980    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl2zf\" (UniqueName: \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-kube-api-access-hl2zf\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.258128 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.258016    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/226f0858-e297-4699-9d2b-59144a762bf1-istiod-ca-cert\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.258128 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.258106    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-certs\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.258306 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.258144    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-istio-token\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.258306 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.258173    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-credential-socket\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.258306 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.258196    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-data\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.258306 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.258207    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-socket\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.258528 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.258457    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-certs\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.258592 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.258569    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-data\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.259027 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.258999    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/226f0858-e297-4699-9d2b-59144a762bf1-istiod-ca-cert\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.260513 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.260486    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-envoy\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.260672 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.260656    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/226f0858-e297-4699-9d2b-59144a762bf1-istio-podinfo\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.319326 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.319297    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-istio-token\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.329170 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.329130    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl2zf\" (UniqueName: \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-kube-api-access-hl2zf\") pod \"e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") " pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.408215 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.408187    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:34.847408 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.847383    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"]
Nov 03 17:11:34.849936 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:11:34.849899    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod226f0858_e297_4699_9d2b_59144a762bf1.slice/crio-33c9b4937d678216828c9078d65d4e42e629b72c20cb349f8d18558076cb533b WatchSource:0}: Error finding container 33c9b4937d678216828c9078d65d4e42e629b72c20cb349f8d18558076cb533b: Status 404 returned error can't find the container with id 33c9b4937d678216828c9078d65d4e42e629b72c20cb349f8d18558076cb533b
Nov 03 17:11:34.852081 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.852045    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:11:34.852171 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.852109    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:11:34.852171 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:34.852138    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:11:35.301196 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:35.301103    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" event={"ID":"226f0858-e297-4699-9d2b-59144a762bf1","Type":"ContainerStarted","Data":"ed365c03ba50439b71878202a20f409d4b92db9d57f72c815e16daf1133a550e"}
Nov 03 17:11:35.301196 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:35.301159    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" event={"ID":"226f0858-e297-4699-9d2b-59144a762bf1","Type":"ContainerStarted","Data":"33c9b4937d678216828c9078d65d4e42e629b72c20cb349f8d18558076cb533b"}
Nov 03 17:11:35.408855 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:35.408822    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:35.414023 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:35.413994    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:35.561847 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:35.560680    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" podStartSLOduration=1.560660955 podStartE2EDuration="1.560660955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:11:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:11:35.428909813 +0000 UTC m=+3581.253361084" watchObservedRunningTime="2026-11-03 17:11:35.560660955 +0000 UTC m=+3581.385112227"
Nov 03 17:11:36.307142 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:36.307112    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:36.307319 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:36.307184    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:11:52.003097 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.003045    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"]
Nov 03 17:11:52.006550 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.006531    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:11:52.129439 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.129399    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-a4764f87-dockercfg-9rcbh\""
Nov 03 17:11:52.129439 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.129434    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-433fd6\""
Nov 03 17:11:52.129882 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.129854    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6qc7\" (UniqueName: \"kubernetes.io/projected/434caf01-dfb4-4243-bf58-f5a88d27c4ba-kube-api-access-h6qc7\") pod \"payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd\" (UID: \"434caf01-dfb4-4243-bf58-f5a88d27c4ba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:11:52.129985 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.129969    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434caf01-dfb4-4243-bf58-f5a88d27c4ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd\" (UID: \"434caf01-dfb4-4243-bf58-f5a88d27c4ba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:11:52.131094 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.131075    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"]
Nov 03 17:11:52.182095 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.182065    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"]
Nov 03 17:11:52.185416 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.185400    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:11:52.220694 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.220661    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"]
Nov 03 17:11:52.230502 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.230476    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t4lv\" (UniqueName: \"kubernetes.io/projected/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-kube-api-access-4t4lv\") pod \"payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s\" (UID: \"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:11:52.230643 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.230534    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6qc7\" (UniqueName: \"kubernetes.io/projected/434caf01-dfb4-4243-bf58-f5a88d27c4ba-kube-api-access-h6qc7\") pod \"payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd\" (UID: \"434caf01-dfb4-4243-bf58-f5a88d27c4ba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:11:52.230643 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.230613    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434caf01-dfb4-4243-bf58-f5a88d27c4ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd\" (UID: \"434caf01-dfb4-4243-bf58-f5a88d27c4ba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:11:52.230721 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.230656    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s\" (UID: \"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:11:52.231594 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.231572    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434caf01-dfb4-4243-bf58-f5a88d27c4ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd\" (UID: \"434caf01-dfb4-4243-bf58-f5a88d27c4ba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:11:52.292437 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.292365    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6qc7\" (UniqueName: \"kubernetes.io/projected/434caf01-dfb4-4243-bf58-f5a88d27c4ba-kube-api-access-h6qc7\") pod \"payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd\" (UID: \"434caf01-dfb4-4243-bf58-f5a88d27c4ba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:11:52.320385 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.320354    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:11:52.331989 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.331966    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s\" (UID: \"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:11:52.332067 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.332056    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4lv\" (UniqueName: \"kubernetes.io/projected/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-kube-api-access-4t4lv\") pod \"payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s\" (UID: \"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:11:52.333070 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.333042    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s\" (UID: \"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:11:52.393289 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.393253    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t4lv\" (UniqueName: \"kubernetes.io/projected/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-kube-api-access-4t4lv\") pod \"payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s\" (UID: \"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:11:52.498472 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.498435    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:11:52.698665 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.698640    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"]
Nov 03 17:11:52.700622 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:11:52.700596    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod434caf01_dfb4_4243_bf58_f5a88d27c4ba.slice/crio-531bc1b7c36a70ed17723e936f842d1abd026b20dbda298ea6677a4c9e5b5185 WatchSource:0}: Error finding container 531bc1b7c36a70ed17723e936f842d1abd026b20dbda298ea6677a4c9e5b5185: Status 404 returned error can't find the container with id 531bc1b7c36a70ed17723e936f842d1abd026b20dbda298ea6677a4c9e5b5185
Nov 03 17:11:52.885604 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:52.885513    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"]
Nov 03 17:11:52.888222 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:11:52.888188    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfb0cb2_3345_4a75_a4f2_99cd5ef34ada.slice/crio-ccc5d2838f2a547773f3d6e8d869104528db531e0507710c7b3461837880724c WatchSource:0}: Error finding container ccc5d2838f2a547773f3d6e8d869104528db531e0507710c7b3461837880724c: Status 404 returned error can't find the container with id ccc5d2838f2a547773f3d6e8d869104528db531e0507710c7b3461837880724c
Nov 03 17:11:53.384839 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:53.384794    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd" event={"ID":"434caf01-dfb4-4243-bf58-f5a88d27c4ba","Type":"ContainerStarted","Data":"114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef"}
Nov 03 17:11:53.385330 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:53.385011    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:11:53.385330 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:53.385055    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd" event={"ID":"434caf01-dfb4-4243-bf58-f5a88d27c4ba","Type":"ContainerStarted","Data":"531bc1b7c36a70ed17723e936f842d1abd026b20dbda298ea6677a4c9e5b5185"}
Nov 03 17:11:53.386490 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:53.386464    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s" event={"ID":"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada","Type":"ContainerStarted","Data":"e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c"}
Nov 03 17:11:53.386653 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:53.386503    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s" event={"ID":"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada","Type":"ContainerStarted","Data":"ccc5d2838f2a547773f3d6e8d869104528db531e0507710c7b3461837880724c"}
Nov 03 17:11:53.386653 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:53.386597    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:11:53.508672 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:53.508620    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd" podStartSLOduration=2.50860608 podStartE2EDuration="2.50860608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:11:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:11:53.507634913 +0000 UTC m=+3599.332086184" watchObservedRunningTime="2026-11-03 17:11:53.50860608 +0000 UTC m=+3599.333057391"
Nov 03 17:11:54.774152 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:54.774117    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:11:54.791971 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:11:54.791947    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:12:03.391451 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:12:03.391414    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:12:04.396085 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:04.396053    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:12:04.396757 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:04.396120    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:12:04.521000 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:04.520939    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s" podStartSLOduration=12.520925451 podStartE2EDuration="12.520925451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:11:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:11:53.638961017 +0000 UTC m=+3599.463412335" watchObservedRunningTime="2026-11-03 17:12:04.520925451 +0000 UTC m=+3610.345376722"
Nov 03 17:12:13.482338 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:13.482259    5605 generic.go:356] "Generic (PLEG): container finished" podID="226f0858-e297-4699-9d2b-59144a762bf1" containerID="ed365c03ba50439b71878202a20f409d4b92db9d57f72c815e16daf1133a550e" exitCode=0
Nov 03 17:12:13.482907 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:13.482338    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" event={"ID":"226f0858-e297-4699-9d2b-59144a762bf1","Type":"ContainerDied","Data":"ed365c03ba50439b71878202a20f409d4b92db9d57f72c815e16daf1133a550e"}
Nov 03 17:12:13.482907 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:13.482846    5605 scope.go:122] "RemoveContainer" containerID="ed365c03ba50439b71878202a20f409d4b92db9d57f72c815e16daf1133a550e"
Nov 03 17:12:13.484018 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:13.483986    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:12:13.484120 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:13.484055    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:12:13.484120 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:13.484089    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:12:14.511216 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:14.511184    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" event={"ID":"226f0858-e297-4699-9d2b-59144a762bf1","Type":"ContainerStarted","Data":"c888049a3344801255128ec472b8c39f8cc8a90b3938ed97d3783b0ba04c1c21"}
Nov 03 17:12:15.409119 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:15.409078    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:12:15.409371 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:15.409345    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Nov 03 17:12:15.409437 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:15.409414    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Nov 03 17:12:16.409104 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:16.409066    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Nov 03 17:12:16.409632 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:16.409130    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Nov 03 17:12:17.409240 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:17.409203    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Nov 03 17:12:17.409674 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:17.409263    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Nov 03 17:12:18.408643 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:18.408606    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Nov 03 17:12:18.408866 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:18.408666    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Nov 03 17:12:19.409391 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:19.409339    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Nov 03 17:12:19.409923 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:19.409404    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Nov 03 17:12:20.413102 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:20.413072    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:12:20.413524 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:20.413341    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:12:20.414217 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:12:20.414201    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:12:33.391311 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:12:33.391223    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:13:03.391627 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:13:03.391588    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:13:33.391223 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:13:33.391185    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:13:40.211467 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:40.211430    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"]
Nov 03 17:13:40.319333 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:40.319300    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"]
Nov 03 17:13:45.212297 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:45.212256    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s" podUID="dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada" containerName="payload-processing" containerID="cri-o://e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c" gracePeriod=25
Nov 03 17:13:45.320430 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:45.320393    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd" podUID="434caf01-dfb4-4243-bf58-f5a88d27c4ba" containerName="payload-pre-processing" containerID="cri-o://114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef" gracePeriod=25
Nov 03 17:13:52.883647 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:52.883609    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"]
Nov 03 17:13:52.884612 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:52.884559    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy" containerID="cri-o://c888049a3344801255128ec472b8c39f8cc8a90b3938ed97d3783b0ba04c1c21" gracePeriod=30
Nov 03 17:13:54.394136 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:54.394092    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd" podUID="434caf01-dfb4-4243-bf58-f5a88d27c4ba" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 17:13:54.394551 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:54.394140    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s" podUID="dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 17:13:58.006681 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.006650    5605 generic.go:356] "Generic (PLEG): container finished" podID="226f0858-e297-4699-9d2b-59144a762bf1" containerID="c888049a3344801255128ec472b8c39f8cc8a90b3938ed97d3783b0ba04c1c21" exitCode=0
Nov 03 17:13:58.007084 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.006736    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" event={"ID":"226f0858-e297-4699-9d2b-59144a762bf1","Type":"ContainerDied","Data":"c888049a3344801255128ec472b8c39f8cc8a90b3938ed97d3783b0ba04c1c21"}
Nov 03 17:13:58.007084 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.006803    5605 scope.go:122] "RemoveContainer" containerID="ed365c03ba50439b71878202a20f409d4b92db9d57f72c815e16daf1133a550e"
Nov 03 17:13:58.145061 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.145039    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:13:58.231057 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.230977    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/226f0858-e297-4699-9d2b-59144a762bf1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/226f0858-e297-4699-9d2b-59144a762bf1-istiod-ca-cert\") pod \"226f0858-e297-4699-9d2b-59144a762bf1\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") "
Nov 03 17:13:58.231057 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231021    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/226f0858-e297-4699-9d2b-59144a762bf1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/226f0858-e297-4699-9d2b-59144a762bf1-istio-podinfo\") pod \"226f0858-e297-4699-9d2b-59144a762bf1\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") "
Nov 03 17:13:58.231293 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231079    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-credential-socket\") pod \"226f0858-e297-4699-9d2b-59144a762bf1\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") "
Nov 03 17:13:58.231293 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231108    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-envoy\") pod \"226f0858-e297-4699-9d2b-59144a762bf1\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") "
Nov 03 17:13:58.231293 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231139    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-istio-token\" (UniqueName: \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-istio-token\") pod \"226f0858-e297-4699-9d2b-59144a762bf1\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") "
Nov 03 17:13:58.231293 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231159    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-certs\") pod \"226f0858-e297-4699-9d2b-59144a762bf1\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") "
Nov 03 17:13:58.231293 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231232    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-kube-api-access-hl2zf\" (UniqueName: \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-kube-api-access-hl2zf\") pod \"226f0858-e297-4699-9d2b-59144a762bf1\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") "
Nov 03 17:13:58.231293 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231261    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-socket\") pod \"226f0858-e297-4699-9d2b-59144a762bf1\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") "
Nov 03 17:13:58.231628 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231297    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-data\") pod \"226f0858-e297-4699-9d2b-59144a762bf1\" (UID: \"226f0858-e297-4699-9d2b-59144a762bf1\") "
Nov 03 17:13:58.231628 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231485    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-credential-socket" pod "226f0858-e297-4699-9d2b-59144a762bf1" (UID: "226f0858-e297-4699-9d2b-59144a762bf1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:13:58.231628 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231509    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/226f0858-e297-4699-9d2b-59144a762bf1-istiod-ca-cert" pod "226f0858-e297-4699-9d2b-59144a762bf1" (UID: "226f0858-e297-4699-9d2b-59144a762bf1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:13:58.231826 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231710    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/226f0858-e297-4699-9d2b-59144a762bf1-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:13:58.231826 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231765    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:13:58.231826 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231711    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-data" pod "226f0858-e297-4699-9d2b-59144a762bf1" (UID: "226f0858-e297-4699-9d2b-59144a762bf1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:13:58.231826 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231768    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-socket" pod "226f0858-e297-4699-9d2b-59144a762bf1" (UID: "226f0858-e297-4699-9d2b-59144a762bf1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:13:58.231826 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.231756    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-certs" pod "226f0858-e297-4699-9d2b-59144a762bf1" (UID: "226f0858-e297-4699-9d2b-59144a762bf1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:13:58.233592 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.233571    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/226f0858-e297-4699-9d2b-59144a762bf1-istio-podinfo" pod "226f0858-e297-4699-9d2b-59144a762bf1" (UID: "226f0858-e297-4699-9d2b-59144a762bf1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 17:13:58.233702 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.233624    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-istio-token" pod "226f0858-e297-4699-9d2b-59144a762bf1" (UID: "226f0858-e297-4699-9d2b-59144a762bf1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:13:58.234039 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.234020    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-envoy" pod "226f0858-e297-4699-9d2b-59144a762bf1" (UID: "226f0858-e297-4699-9d2b-59144a762bf1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:13:58.234107 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.234019    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-kube-api-access-hl2zf" pod "226f0858-e297-4699-9d2b-59144a762bf1" (UID: "226f0858-e297-4699-9d2b-59144a762bf1"). InnerVolumeSpecName "kube-api-access-hl2zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:13:58.332256 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.332226    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/226f0858-e297-4699-9d2b-59144a762bf1-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:13:58.332256 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.332255    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:13:58.332439 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.332266    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:13:58.332439 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.332274    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:13:58.332439 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.332283    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl2zf\" (UniqueName: \"kubernetes.io/projected/226f0858-e297-4699-9d2b-59144a762bf1-kube-api-access-hl2zf\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:13:58.332439 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.332293    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:13:58.332439 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:58.332301    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/226f0858-e297-4699-9d2b-59144a762bf1-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:13:59.012141 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:59.012104    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"
Nov 03 17:13:59.012141 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:59.012121    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8" event={"ID":"226f0858-e297-4699-9d2b-59144a762bf1","Type":"ContainerDied","Data":"33c9b4937d678216828c9078d65d4e42e629b72c20cb349f8d18558076cb533b"}
Nov 03 17:13:59.012836 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:59.012215    5605 scope.go:122] "RemoveContainer" containerID="c888049a3344801255128ec472b8c39f8cc8a90b3938ed97d3783b0ba04c1c21"
Nov 03 17:13:59.235685 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:59.235643    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"]
Nov 03 17:13:59.287229 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:13:59.287149    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-433fd6-openshift-default-846d8c9c65nblg8"]
Nov 03 17:14:00.307450 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:00.307406    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="226f0858-e297-4699-9d2b-59144a762bf1" path="/var/lib/kubelet/pods/226f0858-e297-4699-9d2b-59144a762bf1/volumes"
Nov 03 17:14:03.390425 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:14:03.390387    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:14:04.394624 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:04.394587    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd" podUID="434caf01-dfb4-4243-bf58-f5a88d27c4ba" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 17:14:04.395095 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:04.394601    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s" podUID="dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 17:14:05.464013 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.463989    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:14:05.564438 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.564413    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:14:05.602833 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.602808    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-kube-api-access-4t4lv\" (UniqueName: \"kubernetes.io/projected/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-kube-api-access-4t4lv\") pod \"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada\" (UID: \"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada\") "
Nov 03 17:14:05.602993 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.602899    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-plugins-config-volume\") pod \"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada\" (UID: \"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada\") "
Nov 03 17:14:05.603307 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.603286    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-plugins-config-volume" pod "dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada" (UID: "dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:14:05.605065 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.605044    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-kube-api-access-4t4lv" pod "dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada" (UID: "dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada"). InnerVolumeSpecName "kube-api-access-4t4lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:14:05.703661 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.703627    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/434caf01-dfb4-4243-bf58-f5a88d27c4ba-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434caf01-dfb4-4243-bf58-f5a88d27c4ba-plugins-config-volume\") pod \"434caf01-dfb4-4243-bf58-f5a88d27c4ba\" (UID: \"434caf01-dfb4-4243-bf58-f5a88d27c4ba\") "
Nov 03 17:14:05.703880 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.703778    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/434caf01-dfb4-4243-bf58-f5a88d27c4ba-kube-api-access-h6qc7\" (UniqueName: \"kubernetes.io/projected/434caf01-dfb4-4243-bf58-f5a88d27c4ba-kube-api-access-h6qc7\") pod \"434caf01-dfb4-4243-bf58-f5a88d27c4ba\" (UID: \"434caf01-dfb4-4243-bf58-f5a88d27c4ba\") "
Nov 03 17:14:05.704137 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.704059    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/434caf01-dfb4-4243-bf58-f5a88d27c4ba-plugins-config-volume" pod "434caf01-dfb4-4243-bf58-f5a88d27c4ba" (UID: "434caf01-dfb4-4243-bf58-f5a88d27c4ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:14:05.704322 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.704299    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434caf01-dfb4-4243-bf58-f5a88d27c4ba-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:14:05.704386 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.704328    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t4lv\" (UniqueName: \"kubernetes.io/projected/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-kube-api-access-4t4lv\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:14:05.704386 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.704344    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:14:05.706769 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.706714    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434caf01-dfb4-4243-bf58-f5a88d27c4ba-kube-api-access-h6qc7" pod "434caf01-dfb4-4243-bf58-f5a88d27c4ba" (UID: "434caf01-dfb4-4243-bf58-f5a88d27c4ba"). InnerVolumeSpecName "kube-api-access-h6qc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:14:05.805211 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:05.805123    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6qc7\" (UniqueName: \"kubernetes.io/projected/434caf01-dfb4-4243-bf58-f5a88d27c4ba-kube-api-access-h6qc7\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:14:06.045878 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.045838    5605 generic.go:356] "Generic (PLEG): container finished" podID="dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada" containerID="e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c" exitCode=0
Nov 03 17:14:06.046089 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.045887    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"
Nov 03 17:14:06.046089 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.045897    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s" event={"ID":"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada","Type":"ContainerDied","Data":"e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c"}
Nov 03 17:14:06.046089 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.045996    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s" event={"ID":"dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada","Type":"ContainerDied","Data":"ccc5d2838f2a547773f3d6e8d869104528db531e0507710c7b3461837880724c"}
Nov 03 17:14:06.046089 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.046028    5605 scope.go:122] "RemoveContainer" containerID="e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c"
Nov 03 17:14:06.047445 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.047416    5605 generic.go:356] "Generic (PLEG): container finished" podID="434caf01-dfb4-4243-bf58-f5a88d27c4ba" containerID="114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef" exitCode=0
Nov 03 17:14:06.047571 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.047460    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"
Nov 03 17:14:06.047571 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.047501    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd" event={"ID":"434caf01-dfb4-4243-bf58-f5a88d27c4ba","Type":"ContainerDied","Data":"114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef"}
Nov 03 17:14:06.047667 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.047564    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd" event={"ID":"434caf01-dfb4-4243-bf58-f5a88d27c4ba","Type":"ContainerDied","Data":"531bc1b7c36a70ed17723e936f842d1abd026b20dbda298ea6677a4c9e5b5185"}
Nov 03 17:14:06.055634 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.055616    5605 scope.go:122] "RemoveContainer" containerID="e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c"
Nov 03 17:14:06.055942 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:14:06.055919    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c\": container with ID starting with e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c not found: ID does not exist" containerID="e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c"
Nov 03 17:14:06.056084 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.055947    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c"} err="failed to get container status \"e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c\": rpc error: code = NotFound desc = could not find container \"e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c\": container with ID starting with e43ed2ef8395587ab7addf3b41ba3bd9c22ad3d9dc43f6fbfe0cde3b2961159c not found: ID does not exist"
Nov 03 17:14:06.056084 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.055964    5605 scope.go:122] "RemoveContainer" containerID="114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef"
Nov 03 17:14:06.063635 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.063618    5605 scope.go:122] "RemoveContainer" containerID="114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef"
Nov 03 17:14:06.063920 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:14:06.063903    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef\": container with ID starting with 114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef not found: ID does not exist" containerID="114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef"
Nov 03 17:14:06.063982 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.063927    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef"} err="failed to get container status \"114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef\": rpc error: code = NotFound desc = could not find container \"114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef\": container with ID starting with 114f6de690e49d436e676640eca78e0e19bbf2b04df7f1d6d53f76605da23bef not found: ID does not exist"
Nov 03 17:14:06.218339 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.218304    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"]
Nov 03 17:14:06.269455 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.269419    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-433fd6-565647669fxq9s"]
Nov 03 17:14:06.307283 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.307200    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada" path="/var/lib/kubelet/pods/dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada/volumes"
Nov 03 17:14:06.383681 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.383646    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"]
Nov 03 17:14:06.390673 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:06.390651    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-433fd6-7c49cbzmhd"]
Nov 03 17:14:08.307280 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:08.307246    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="434caf01-dfb4-4243-bf58-f5a88d27c4ba" path="/var/lib/kubelet/pods/434caf01-dfb4-4243-bf58-f5a88d27c4ba/volumes"
Nov 03 17:14:11.126539 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:11.126497    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rn227 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:14:11.127344 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:14:11.127304    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" podUID="b0246a86-f897-42f0-88eb-38c00c8b1bc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:14:33.391255 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:14:33.391215    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:15:03.391659 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:15:03.391618    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:15:33.391865 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:15:33.391770    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:16:03.392705 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:16:03.392668    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:16:33.391765 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:16:33.391703    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:16:49.756959 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.756929    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"]
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757282    5605 cpu_manager.go:414] "containerMap: removing container" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy"
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757293    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy"
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757313    5605 cpu_manager.go:414] "containerMap: removing container" podUID="dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada" containerName="payload-processing"
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757318    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada" containerName="payload-processing"
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757335    5605 cpu_manager.go:414] "containerMap: removing container" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy"
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757341    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy"
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757347    5605 cpu_manager.go:414] "containerMap: removing container" podUID="434caf01-dfb4-4243-bf58-f5a88d27c4ba" containerName="payload-pre-processing"
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757352    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="434caf01-dfb4-4243-bf58-f5a88d27c4ba" containerName="payload-pre-processing"
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757407    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcfb0cb2-3345-4a75-a4f2-99cd5ef34ada" containerName="payload-processing"
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757418    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy"
Nov 03 17:16:49.757448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.757422    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="434caf01-dfb4-4243-bf58-f5a88d27c4ba" containerName="payload-pre-processing"
Nov 03 17:16:49.760476 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.760460    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.820174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.820144    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-a56144-openshift-default-dockercfg-hm2db\""
Nov 03 17:16:49.874696 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.874659    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"]
Nov 03 17:16:49.881953 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.881917    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-token\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.882144 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.882067    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-podinfo\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.882144 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.882118    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9caeefb5-d561-4f27-9525-3a1d6418bdae-istiod-ca-cert\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.882240 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.882150    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-envoy\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.882315 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.882294    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-socket\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.882359 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.882331    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kffgf\" (UniqueName: \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-kube-api-access-kffgf\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.882417 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.882354    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-certs\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.882473 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.882418    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-credential-socket\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.882528 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.882478    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-data\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.983422 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.983373    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-credential-socket\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.983422 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.983429    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-data\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.983699 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.983495    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-token\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.983699 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.983568    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-podinfo\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.983699 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.983595    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9caeefb5-d561-4f27-9525-3a1d6418bdae-istiod-ca-cert\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.983699 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.983620    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-envoy\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.983699 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.983662    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-socket\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.983699 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.983694    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kffgf\" (UniqueName: \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-kube-api-access-kffgf\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.984068 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.983718    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-certs\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.984068 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.983933    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-data\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.984189 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.984107    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-certs\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.984488 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.984455    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9caeefb5-d561-4f27-9525-3a1d6418bdae-istiod-ca-cert\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.984955 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.984699    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-credential-socket\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.985138 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.985104    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-socket\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.986563 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.986541    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-envoy\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:49.987221 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:49.987202    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-podinfo\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:50.047387 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:50.047315    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-token\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:50.047535 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:50.047412    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kffgf\" (UniqueName: \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-kube-api-access-kffgf\") pod \"e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") " pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:50.074232 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:50.073949    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:50.510720 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:50.510645    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"]
Nov 03 17:16:50.513222 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:16:50.513191    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9caeefb5_d561_4f27_9525_3a1d6418bdae.slice/crio-82e9fc7ce99b6f41f9a1333fbb3a527232421c9cf379efaf8a927750766247a6 WatchSource:0}: Error finding container 82e9fc7ce99b6f41f9a1333fbb3a527232421c9cf379efaf8a927750766247a6: Status 404 returned error can't find the container with id 82e9fc7ce99b6f41f9a1333fbb3a527232421c9cf379efaf8a927750766247a6
Nov 03 17:16:50.515425 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:50.515392    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:16:50.515490 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:50.515467    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:16:50.515535 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:50.515494    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:16:50.805652 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:50.805562    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" event={"ID":"9caeefb5-d561-4f27-9525-3a1d6418bdae","Type":"ContainerStarted","Data":"de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29"}
Nov 03 17:16:50.805652 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:50.805618    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" event={"ID":"9caeefb5-d561-4f27-9525-3a1d6418bdae","Type":"ContainerStarted","Data":"82e9fc7ce99b6f41f9a1333fbb3a527232421c9cf379efaf8a927750766247a6"}
Nov 03 17:16:50.940138 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:50.940078    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" podStartSLOduration=1.94006291 podStartE2EDuration="1.94006291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:16:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:16:50.938226582 +0000 UTC m=+3896.762677853" watchObservedRunningTime="2026-11-03 17:16:50.94006291 +0000 UTC m=+3896.764514180"
Nov 03 17:16:51.074675 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:51.074591    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:51.079827 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:51.079805    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:51.810064 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:51.810028    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:51.810997 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:51.810978    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:16:54.802226 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:54.802198    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:16:54.821129 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:54.821101    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:16:57.447123 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.447089    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"]
Nov 03 17:16:57.447513 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.447494    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="226f0858-e297-4699-9d2b-59144a762bf1" containerName="istio-proxy"
Nov 03 17:16:57.450440 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.450423    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:16:57.509132 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.509095    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"]
Nov 03 17:16:57.555482 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.555444    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d7f8\" (UniqueName: \"kubernetes.io/projected/d5625c67-b99e-405f-960c-3fa8c904eaa3-kube-api-access-2d7f8\") pod \"payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4\" (UID: \"d5625c67-b99e-405f-960c-3fa8c904eaa3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:16:57.555692 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.555518    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5625c67-b99e-405f-960c-3fa8c904eaa3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4\" (UID: \"d5625c67-b99e-405f-960c-3fa8c904eaa3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:16:57.558856 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.558828    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-a56144\""
Nov 03 17:16:57.559205 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.559181    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-78a86052-dockercfg-zhq75\""
Nov 03 17:16:57.611315 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.611283    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"]
Nov 03 17:16:57.614691 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.614667    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:16:57.654516 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.654482    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"]
Nov 03 17:16:57.656358 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.656326    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5625c67-b99e-405f-960c-3fa8c904eaa3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4\" (UID: \"d5625c67-b99e-405f-960c-3fa8c904eaa3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:16:57.656493 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.656458    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d7f8\" (UniqueName: \"kubernetes.io/projected/d5625c67-b99e-405f-960c-3fa8c904eaa3-kube-api-access-2d7f8\") pod \"payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4\" (UID: \"d5625c67-b99e-405f-960c-3fa8c904eaa3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:16:57.657019 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.657001    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5625c67-b99e-405f-960c-3fa8c904eaa3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4\" (UID: \"d5625c67-b99e-405f-960c-3fa8c904eaa3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:16:57.717246 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.717164    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d7f8\" (UniqueName: \"kubernetes.io/projected/d5625c67-b99e-405f-960c-3fa8c904eaa3-kube-api-access-2d7f8\") pod \"payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4\" (UID: \"d5625c67-b99e-405f-960c-3fa8c904eaa3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:16:57.757690 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.757653    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f713fc70-e376-4e90-8b71-afd253286c4c-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg\" (UID: \"f713fc70-e376-4e90-8b71-afd253286c4c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:16:57.757877 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.757699    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v92nq\" (UniqueName: \"kubernetes.io/projected/f713fc70-e376-4e90-8b71-afd253286c4c-kube-api-access-v92nq\") pod \"payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg\" (UID: \"f713fc70-e376-4e90-8b71-afd253286c4c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:16:57.762509 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.762485    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:16:57.858861 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.858831    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f713fc70-e376-4e90-8b71-afd253286c4c-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg\" (UID: \"f713fc70-e376-4e90-8b71-afd253286c4c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:16:57.859017 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.858889    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v92nq\" (UniqueName: \"kubernetes.io/projected/f713fc70-e376-4e90-8b71-afd253286c4c-kube-api-access-v92nq\") pod \"payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg\" (UID: \"f713fc70-e376-4e90-8b71-afd253286c4c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:16:57.859542 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.859523    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f713fc70-e376-4e90-8b71-afd253286c4c-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg\" (UID: \"f713fc70-e376-4e90-8b71-afd253286c4c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:16:57.920002 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.919964    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v92nq\" (UniqueName: \"kubernetes.io/projected/f713fc70-e376-4e90-8b71-afd253286c4c-kube-api-access-v92nq\") pod \"payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg\" (UID: \"f713fc70-e376-4e90-8b71-afd253286c4c\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:16:57.926771 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:57.926744    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:16:58.150152 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:58.150125    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"]
Nov 03 17:16:58.152317 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:16:58.152288    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5625c67_b99e_405f_960c_3fa8c904eaa3.slice/crio-67c5f5a9d70f6fa50c080e4a76a04a21bcc8943e8425b44b4e4536aede57f11c WatchSource:0}: Error finding container 67c5f5a9d70f6fa50c080e4a76a04a21bcc8943e8425b44b4e4536aede57f11c: Status 404 returned error can't find the container with id 67c5f5a9d70f6fa50c080e4a76a04a21bcc8943e8425b44b4e4536aede57f11c
Nov 03 17:16:58.307047 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:58.307026    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"]
Nov 03 17:16:58.308402 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:16:58.308372    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf713fc70_e376_4e90_8b71_afd253286c4c.slice/crio-e082ed7be31de25289455c7c61acf0eefd61f43bc5929d2a4e5f89ab46962f1f WatchSource:0}: Error finding container e082ed7be31de25289455c7c61acf0eefd61f43bc5929d2a4e5f89ab46962f1f: Status 404 returned error can't find the container with id e082ed7be31de25289455c7c61acf0eefd61f43bc5929d2a4e5f89ab46962f1f
Nov 03 17:16:58.843106 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:58.843069    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4" event={"ID":"d5625c67-b99e-405f-960c-3fa8c904eaa3","Type":"ContainerStarted","Data":"8ceacab07d9f71a59431aff19eaaf65262c8757d47787693337a6832a61607ef"}
Nov 03 17:16:58.843577 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:58.843135    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4" event={"ID":"d5625c67-b99e-405f-960c-3fa8c904eaa3","Type":"ContainerStarted","Data":"67c5f5a9d70f6fa50c080e4a76a04a21bcc8943e8425b44b4e4536aede57f11c"}
Nov 03 17:16:58.843577 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:58.843270    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:16:58.844786 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:58.844756    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg" event={"ID":"f713fc70-e376-4e90-8b71-afd253286c4c","Type":"ContainerStarted","Data":"0460c29c116e726138074a71978ee192504942129e2e2bfe12f254b7d1104c2e"}
Nov 03 17:16:58.844925 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:58.844805    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg" event={"ID":"f713fc70-e376-4e90-8b71-afd253286c4c","Type":"ContainerStarted","Data":"e082ed7be31de25289455c7c61acf0eefd61f43bc5929d2a4e5f89ab46962f1f"}
Nov 03 17:16:58.844925 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:58.844826    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:16:58.965438 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:16:58.965385    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4" podStartSLOduration=1.965371883 podStartE2EDuration="1.965371883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:16:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:16:58.963857823 +0000 UTC m=+3904.788309094" watchObservedRunningTime="2026-11-03 17:16:58.965371883 +0000 UTC m=+3904.789823157"
Nov 03 17:17:03.390230 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:17:03.390190    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:17:09.852962 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:09.852926    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:17:09.853384 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:09.853002    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:17:09.977053 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:09.976999    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg" podStartSLOduration=12.976985779 podStartE2EDuration="12.976985779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:16:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:16:59.088492855 +0000 UTC m=+3904.912944131" watchObservedRunningTime="2026-11-03 17:17:09.976985779 +0000 UTC m=+3915.801437041"
Nov 03 17:17:17.948986 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:17.948950    5605 generic.go:356] "Generic (PLEG): container finished" podID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerID="de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29" exitCode=0
Nov 03 17:17:17.949544 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:17.949026    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" event={"ID":"9caeefb5-d561-4f27-9525-3a1d6418bdae","Type":"ContainerDied","Data":"de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29"}
Nov 03 17:17:17.949544 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:17.949532    5605 scope.go:122] "RemoveContainer" containerID="de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29"
Nov 03 17:17:17.950837 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:17.950810    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:17:17.950916 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:17.950875    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:17:17.950916 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:17.950909    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:17:18.955715 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:18.955679    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" event={"ID":"9caeefb5-d561-4f27-9525-3a1d6418bdae","Type":"ContainerStarted","Data":"ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730"}
Nov 03 17:17:19.074818 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:19.074782    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:17:19.076189 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:19.076159    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Nov 03 17:17:19.076294 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:19.076218    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Nov 03 17:17:20.074647 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:20.074614    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Nov 03 17:17:20.075098 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:20.074675    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Nov 03 17:17:21.074392 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:21.074355    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Nov 03 17:17:21.074658 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:21.074425    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Nov 03 17:17:21.810342 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:21.810291    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:17:22.075346 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:22.075263    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Nov 03 17:17:22.075346 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:22.075315    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Nov 03 17:17:23.074600 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:23.074563    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused" start-of-body=
Nov 03 17:17:23.075206 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:23.074628    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": dial tcp 10.133.0.154:15021: connect: connection refused"
Nov 03 17:17:24.078535 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:24.078502    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:17:24.079584 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:17:24.079563    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:17:33.391357 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:17:33.391315    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:18:03.394947 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:18:03.394908    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:18:20.058823 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:20.058781    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"]
Nov 03 17:18:20.165978 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:20.165944    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"]
Nov 03 17:18:25.060103 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:25.060015    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg" podUID="f713fc70-e376-4e90-8b71-afd253286c4c" containerName="payload-processing" containerID="cri-o://0460c29c116e726138074a71978ee192504942129e2e2bfe12f254b7d1104c2e" gracePeriod=25
Nov 03 17:18:25.166737 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:25.166697    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4" podUID="d5625c67-b99e-405f-960c-3fa8c904eaa3" containerName="payload-pre-processing" containerID="cri-o://8ceacab07d9f71a59431aff19eaaf65262c8757d47787693337a6832a61607ef" gracePeriod=25
Nov 03 17:18:29.851596 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:29.851516    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg" podUID="f713fc70-e376-4e90-8b71-afd253286c4c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 17:18:29.851596 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:29.851539    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4" podUID="d5625c67-b99e-405f-960c-3fa8c904eaa3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 17:18:33.391840 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:18:33.391807    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:18:33.908332 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:33.908291    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"]
Nov 03 17:18:33.908584 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:33.908555    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy" containerID="cri-o://ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730" gracePeriod=30
Nov 03 17:18:39.164838 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.164801    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:18:39.206124 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206071    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-data\") pod \"9caeefb5-d561-4f27-9525-3a1d6418bdae\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") "
Nov 03 17:18:39.206124 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206106    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-credential-socket\") pod \"9caeefb5-d561-4f27-9525-3a1d6418bdae\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") "
Nov 03 17:18:39.206291 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206146    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9caeefb5-d561-4f27-9525-3a1d6418bdae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9caeefb5-d561-4f27-9525-3a1d6418bdae-istiod-ca-cert\") pod \"9caeefb5-d561-4f27-9525-3a1d6418bdae\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") "
Nov 03 17:18:39.206291 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206201    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-socket\") pod \"9caeefb5-d561-4f27-9525-3a1d6418bdae\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") "
Nov 03 17:18:39.206417 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206334    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-data" pod "9caeefb5-d561-4f27-9525-3a1d6418bdae" (UID: "9caeefb5-d561-4f27-9525-3a1d6418bdae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:18:39.206417 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206357    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-credential-socket" pod "9caeefb5-d561-4f27-9525-3a1d6418bdae" (UID: "9caeefb5-d561-4f27-9525-3a1d6418bdae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:18:39.206417 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206378    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-certs\") pod \"9caeefb5-d561-4f27-9525-3a1d6418bdae\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") "
Nov 03 17:18:39.206596 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206425    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-token\" (UniqueName: \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-token\") pod \"9caeefb5-d561-4f27-9525-3a1d6418bdae\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") "
Nov 03 17:18:39.206596 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206470    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-envoy\") pod \"9caeefb5-d561-4f27-9525-3a1d6418bdae\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") "
Nov 03 17:18:39.206596 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206486    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-socket" pod "9caeefb5-d561-4f27-9525-3a1d6418bdae" (UID: "9caeefb5-d561-4f27-9525-3a1d6418bdae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:18:39.206596 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206514    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-podinfo\") pod \"9caeefb5-d561-4f27-9525-3a1d6418bdae\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") "
Nov 03 17:18:39.206596 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206564    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-kube-api-access-kffgf\" (UniqueName: \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-kube-api-access-kffgf\") pod \"9caeefb5-d561-4f27-9525-3a1d6418bdae\" (UID: \"9caeefb5-d561-4f27-9525-3a1d6418bdae\") "
Nov 03 17:18:39.206886 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206604    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9caeefb5-d561-4f27-9525-3a1d6418bdae-istiod-ca-cert" pod "9caeefb5-d561-4f27-9525-3a1d6418bdae" (UID: "9caeefb5-d561-4f27-9525-3a1d6418bdae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:18:39.206943 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206891    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-certs" pod "9caeefb5-d561-4f27-9525-3a1d6418bdae" (UID: "9caeefb5-d561-4f27-9525-3a1d6418bdae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:18:39.207000 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206961    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:39.207000 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206983    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:39.207000 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.206997    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9caeefb5-d561-4f27-9525-3a1d6418bdae-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:39.207338 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.207012    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:39.208940 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.208910    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-token" pod "9caeefb5-d561-4f27-9525-3a1d6418bdae" (UID: "9caeefb5-d561-4f27-9525-3a1d6418bdae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:18:39.209047 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.208938    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-kube-api-access-kffgf" pod "9caeefb5-d561-4f27-9525-3a1d6418bdae" (UID: "9caeefb5-d561-4f27-9525-3a1d6418bdae"). InnerVolumeSpecName "kube-api-access-kffgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:18:39.209047 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.209026    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-envoy" pod "9caeefb5-d561-4f27-9525-3a1d6418bdae" (UID: "9caeefb5-d561-4f27-9525-3a1d6418bdae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:18:39.209047 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.209029    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-podinfo" pod "9caeefb5-d561-4f27-9525-3a1d6418bdae" (UID: "9caeefb5-d561-4f27-9525-3a1d6418bdae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 17:18:39.308008 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.307954    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:39.308008 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.307999    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:39.308008 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.308009    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:39.308008 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.308018    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9caeefb5-d561-4f27-9525-3a1d6418bdae-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:39.308008 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.308028    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kffgf\" (UniqueName: \"kubernetes.io/projected/9caeefb5-d561-4f27-9525-3a1d6418bdae-kube-api-access-kffgf\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:39.317385 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.317344    5605 generic.go:356] "Generic (PLEG): container finished" podID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerID="ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730" exitCode=0
Nov 03 17:18:39.317565 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.317406    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"
Nov 03 17:18:39.317565 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.317404    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" event={"ID":"9caeefb5-d561-4f27-9525-3a1d6418bdae","Type":"ContainerDied","Data":"ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730"}
Nov 03 17:18:39.317565 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.317525    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" event={"ID":"9caeefb5-d561-4f27-9525-3a1d6418bdae","Type":"ContainerDied","Data":"82e9fc7ce99b6f41f9a1333fbb3a527232421c9cf379efaf8a927750766247a6"}
Nov 03 17:18:39.317565 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.317553    5605 scope.go:122] "RemoveContainer" containerID="ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730"
Nov 03 17:18:39.327930 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.327909    5605 scope.go:122] "RemoveContainer" containerID="de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29"
Nov 03 17:18:39.336500 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.336480    5605 scope.go:122] "RemoveContainer" containerID="ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730"
Nov 03 17:18:39.336821 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:18:39.336800    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730\": container with ID starting with ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730 not found: ID does not exist" containerID="ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730"
Nov 03 17:18:39.336911 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.336832    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730"} err="failed to get container status \"ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730\": rpc error: code = NotFound desc = could not find container \"ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730\": container with ID starting with ba7b27dbeead06c1160d64c0b3d49b128571c8fbaa999519534ea885e0c1b730 not found: ID does not exist"
Nov 03 17:18:39.336911 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.336849    5605 scope.go:122] "RemoveContainer" containerID="de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29"
Nov 03 17:18:39.337107 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:18:39.337089    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29\": container with ID starting with de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29 not found: ID does not exist" containerID="de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29"
Nov 03 17:18:39.337148 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.337112    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29"} err="failed to get container status \"de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29\": rpc error: code = NotFound desc = could not find container \"de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29\": container with ID starting with de5e89c480c988ba1304707ed9b522701c2ff8f73cf6b3bb77b31d93e470dd29 not found: ID does not exist"
Nov 03 17:18:39.455954 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.455919    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"]
Nov 03 17:18:39.462146 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.462076    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd"]
Nov 03 17:18:39.851473 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.851439    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg" podUID="f713fc70-e376-4e90-8b71-afd253286c4c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 17:18:39.851473 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:39.851467    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4" podUID="d5625c67-b99e-405f-960c-3fa8c904eaa3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 17:18:40.079041 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:40.079005    5605 patch_prober.go:29] interesting pod/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.154:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:18:40.079228 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:40.079070    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-a56144-openshift-default-648dd99757z77fd" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.154:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:18:40.313393 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:40.313317    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" path="/var/lib/kubelet/pods/9caeefb5-d561-4f27-9525-3a1d6418bdae/volumes"
Nov 03 17:18:45.349522 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.349316    5605 generic.go:356] "Generic (PLEG): container finished" podID="d5625c67-b99e-405f-960c-3fa8c904eaa3" containerID="8ceacab07d9f71a59431aff19eaaf65262c8757d47787693337a6832a61607ef" exitCode=0
Nov 03 17:18:45.349522 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.349443    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4" event={"ID":"d5625c67-b99e-405f-960c-3fa8c904eaa3","Type":"ContainerDied","Data":"8ceacab07d9f71a59431aff19eaaf65262c8757d47787693337a6832a61607ef"}
Nov 03 17:18:45.353459 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.353288    5605 generic.go:356] "Generic (PLEG): container finished" podID="f713fc70-e376-4e90-8b71-afd253286c4c" containerID="0460c29c116e726138074a71978ee192504942129e2e2bfe12f254b7d1104c2e" exitCode=0
Nov 03 17:18:45.353459 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.353389    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg" event={"ID":"f713fc70-e376-4e90-8b71-afd253286c4c","Type":"ContainerDied","Data":"0460c29c116e726138074a71978ee192504942129e2e2bfe12f254b7d1104c2e"}
Nov 03 17:18:45.399113 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.397098    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:18:45.469441 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.468795    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f713fc70-e376-4e90-8b71-afd253286c4c-kube-api-access-v92nq\" (UniqueName: \"kubernetes.io/projected/f713fc70-e376-4e90-8b71-afd253286c4c-kube-api-access-v92nq\") pod \"f713fc70-e376-4e90-8b71-afd253286c4c\" (UID: \"f713fc70-e376-4e90-8b71-afd253286c4c\") "
Nov 03 17:18:45.469441 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.468926    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f713fc70-e376-4e90-8b71-afd253286c4c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f713fc70-e376-4e90-8b71-afd253286c4c-plugins-config-volume\") pod \"f713fc70-e376-4e90-8b71-afd253286c4c\" (UID: \"f713fc70-e376-4e90-8b71-afd253286c4c\") "
Nov 03 17:18:45.469996 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.469940    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f713fc70-e376-4e90-8b71-afd253286c4c-plugins-config-volume" pod "f713fc70-e376-4e90-8b71-afd253286c4c" (UID: "f713fc70-e376-4e90-8b71-afd253286c4c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:18:45.472511 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.472481    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f713fc70-e376-4e90-8b71-afd253286c4c-kube-api-access-v92nq" pod "f713fc70-e376-4e90-8b71-afd253286c4c" (UID: "f713fc70-e376-4e90-8b71-afd253286c4c"). InnerVolumeSpecName "kube-api-access-v92nq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:18:45.486653 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.486623    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:18:45.570149 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.570093    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5625c67-b99e-405f-960c-3fa8c904eaa3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5625c67-b99e-405f-960c-3fa8c904eaa3-plugins-config-volume\") pod \"d5625c67-b99e-405f-960c-3fa8c904eaa3\" (UID: \"d5625c67-b99e-405f-960c-3fa8c904eaa3\") "
Nov 03 17:18:45.570366 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.570207    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5625c67-b99e-405f-960c-3fa8c904eaa3-kube-api-access-2d7f8\" (UniqueName: \"kubernetes.io/projected/d5625c67-b99e-405f-960c-3fa8c904eaa3-kube-api-access-2d7f8\") pod \"d5625c67-b99e-405f-960c-3fa8c904eaa3\" (UID: \"d5625c67-b99e-405f-960c-3fa8c904eaa3\") "
Nov 03 17:18:45.570583 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.570557    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v92nq\" (UniqueName: \"kubernetes.io/projected/f713fc70-e376-4e90-8b71-afd253286c4c-kube-api-access-v92nq\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:45.570663 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.570590    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f713fc70-e376-4e90-8b71-afd253286c4c-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:45.570663 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.570595    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5625c67-b99e-405f-960c-3fa8c904eaa3-plugins-config-volume" pod "d5625c67-b99e-405f-960c-3fa8c904eaa3" (UID: "d5625c67-b99e-405f-960c-3fa8c904eaa3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:18:45.573163 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.573130    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5625c67-b99e-405f-960c-3fa8c904eaa3-kube-api-access-2d7f8" pod "d5625c67-b99e-405f-960c-3fa8c904eaa3" (UID: "d5625c67-b99e-405f-960c-3fa8c904eaa3"). InnerVolumeSpecName "kube-api-access-2d7f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:18:45.671772 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.671654    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d5625c67-b99e-405f-960c-3fa8c904eaa3-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:45.671772 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:45.671694    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d7f8\" (UniqueName: \"kubernetes.io/projected/d5625c67-b99e-405f-960c-3fa8c904eaa3-kube-api-access-2d7f8\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:18:46.423795 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:46.423751    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4" event={"ID":"d5625c67-b99e-405f-960c-3fa8c904eaa3","Type":"ContainerDied","Data":"67c5f5a9d70f6fa50c080e4a76a04a21bcc8943e8425b44b4e4536aede57f11c"}
Nov 03 17:18:46.424414 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:46.424389    5605 scope.go:122] "RemoveContainer" containerID="8ceacab07d9f71a59431aff19eaaf65262c8757d47787693337a6832a61607ef"
Nov 03 17:18:46.438768 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:46.430203    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"
Nov 03 17:18:46.444863 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:46.444845    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"
Nov 03 17:18:46.446400 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:46.445417    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg" event={"ID":"f713fc70-e376-4e90-8b71-afd253286c4c","Type":"ContainerDied","Data":"e082ed7be31de25289455c7c61acf0eefd61f43bc5929d2a4e5f89ab46962f1f"}
Nov 03 17:18:46.464215 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:46.464189    5605 scope.go:122] "RemoveContainer" containerID="0460c29c116e726138074a71978ee192504942129e2e2bfe12f254b7d1104c2e"
Nov 03 17:18:46.616338 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:46.616291    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"]
Nov 03 17:18:46.669810 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:46.669175    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a56144-647dd7jzk4"]
Nov 03 17:18:46.760314 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:46.760278    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"]
Nov 03 17:18:46.762922 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:46.762896    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a56144-b6699cb4fg6gcg"]
Nov 03 17:18:47.169863 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:47.169827    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:18:47.170166 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:47.170135    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:18:48.306943 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:48.306913    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5625c67-b99e-405f-960c-3fa8c904eaa3" path="/var/lib/kubelet/pods/d5625c67-b99e-405f-960c-3fa8c904eaa3/volumes"
Nov 03 17:18:48.307365 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:18:48.307289    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f713fc70-e376-4e90-8b71-afd253286c4c" path="/var/lib/kubelet/pods/f713fc70-e376-4e90-8b71-afd253286c4c/volumes"
Nov 03 17:19:03.391468 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:19:03.391430    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:19:23.346034 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346002    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"]
Nov 03 17:19:23.346458 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346351    5605 cpu_manager.go:414] "containerMap: removing container" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy"
Nov 03 17:19:23.346458 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346362    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy"
Nov 03 17:19:23.346458 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346376    5605 cpu_manager.go:414] "containerMap: removing container" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy"
Nov 03 17:19:23.346458 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346381    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy"
Nov 03 17:19:23.346458 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346401    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f713fc70-e376-4e90-8b71-afd253286c4c" containerName="payload-processing"
Nov 03 17:19:23.346458 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346407    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f713fc70-e376-4e90-8b71-afd253286c4c" containerName="payload-processing"
Nov 03 17:19:23.346458 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346417    5605 cpu_manager.go:414] "containerMap: removing container" podUID="d5625c67-b99e-405f-960c-3fa8c904eaa3" containerName="payload-pre-processing"
Nov 03 17:19:23.346458 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346422    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5625c67-b99e-405f-960c-3fa8c904eaa3" containerName="payload-pre-processing"
Nov 03 17:19:23.346757 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346479    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy"
Nov 03 17:19:23.346757 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346487    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="9caeefb5-d561-4f27-9525-3a1d6418bdae" containerName="istio-proxy"
Nov 03 17:19:23.346757 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346496    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f713fc70-e376-4e90-8b71-afd253286c4c" containerName="payload-processing"
Nov 03 17:19:23.346757 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.346503    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5625c67-b99e-405f-960c-3fa8c904eaa3" containerName="payload-pre-processing"
Nov 03 17:19:23.350941 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.350923    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.455278 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.455247    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"]
Nov 03 17:19:23.459592 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.459568    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-dd9d4320-kserve-self-signed-certs\""
Nov 03 17:19:23.469062 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.469037    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d109a93-d360-400d-a44b-453f14b06575-tls-certs\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.469197 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.469075    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-tmp-dir\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.469197 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.469109    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-home\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.469282 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.469259    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-model-cache\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.469322 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.469294    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmcmc\" (UniqueName: \"kubernetes.io/projected/8d109a93-d360-400d-a44b-453f14b06575-kube-api-access-tmcmc\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.469362 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.469323    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-dshm\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.570808 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.570774    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-home\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.570808 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.570820    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-model-cache\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.571054 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.570843    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmcmc\" (UniqueName: \"kubernetes.io/projected/8d109a93-d360-400d-a44b-453f14b06575-kube-api-access-tmcmc\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.571054 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.570888    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-dshm\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.571054 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.570971    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d109a93-d360-400d-a44b-453f14b06575-tls-certs\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.571054 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.571008    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-tmp-dir\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.571299 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.571273    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-home\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.571448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.571333    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-model-cache\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.571448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.571380    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-tmp-dir\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.573378 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.573353    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-dshm\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.573590 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.573570    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d109a93-d360-400d-a44b-453f14b06575-tls-certs\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.631625 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.631563    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmcmc\" (UniqueName: \"kubernetes.io/projected/8d109a93-d360-400d-a44b-453f14b06575-kube-api-access-tmcmc\") pod \"e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") " pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:23.661828 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:23.661802    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:24.072870 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:24.072845    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"]
Nov 03 17:19:24.075070 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:19:24.075032    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d109a93_d360_400d_a44b_453f14b06575.slice/crio-42af2373edf94964067443197f92e9cbbd09e72e492894a4f17b091f42d74f7d WatchSource:0}: Error finding container 42af2373edf94964067443197f92e9cbbd09e72e492894a4f17b091f42d74f7d: Status 404 returned error can't find the container with id 42af2373edf94964067443197f92e9cbbd09e72e492894a4f17b091f42d74f7d
Nov 03 17:19:24.644373 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:24.644339    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5" event={"ID":"8d109a93-d360-400d-a44b-453f14b06575","Type":"ContainerStarted","Data":"9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586"}
Nov 03 17:19:24.644810 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:24.644381    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5" event={"ID":"8d109a93-d360-400d-a44b-453f14b06575","Type":"ContainerStarted","Data":"42af2373edf94964067443197f92e9cbbd09e72e492894a4f17b091f42d74f7d"}
Nov 03 17:19:24.662811 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:24.662781    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:24.771556 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:24.771486    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5" podStartSLOduration=1.771472347 podStartE2EDuration="1.771472347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:19:24.770985622 +0000 UTC m=+4050.595436893" watchObservedRunningTime="2026-11-03 17:19:24.771472347 +0000 UTC m=+4050.595923614"
Nov 03 17:19:31.366753 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:31.363854    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-zgbzx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:19:31.366753 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:31.363923    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podUID="04b8d456-e273-4b57-94b7-7bffc3831f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:19:33.390624 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:19:33.390590    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:19:33.663086 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:33.662994    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:33.675321 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:33.675294    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:19:33.717482 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:19:33.717458    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:20:01.376956 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:01.376850    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-zgbzx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:20:01.377877 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:01.377820    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podUID="04b8d456-e273-4b57-94b7-7bffc3831f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:20:02.142920 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:02.142869    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-gjzpm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:20:02.143265 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:02.143216    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gjzpm" podUID="dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:20:03.391457 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:20:03.391419    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:20:12.992974 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:12.992940    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"]
Nov 03 17:20:16.360916 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:16.360862    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-zgbzx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:20:16.361600 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:16.361564    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podUID="04b8d456-e273-4b57-94b7-7bffc3831f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:20:28.013716 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.013668    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5" podUID="8d109a93-d360-400d-a44b-453f14b06575" containerName="main" containerID="cri-o://9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586" gracePeriod=45
Nov 03 17:20:28.256773 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.256747    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:20:28.374624 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.374593    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-home\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-home\") pod \"8d109a93-d360-400d-a44b-453f14b06575\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") "
Nov 03 17:20:28.374852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.374635    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-model-cache\") pod \"8d109a93-d360-400d-a44b-453f14b06575\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") "
Nov 03 17:20:28.374852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.374671    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-dshm\") pod \"8d109a93-d360-400d-a44b-453f14b06575\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") "
Nov 03 17:20:28.374852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.374749    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d109a93-d360-400d-a44b-453f14b06575-tls-certs\" (UniqueName: \"kubernetes.io/secret/8d109a93-d360-400d-a44b-453f14b06575-tls-certs\") pod \"8d109a93-d360-400d-a44b-453f14b06575\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") "
Nov 03 17:20:28.374852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.374801    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d109a93-d360-400d-a44b-453f14b06575-kube-api-access-tmcmc\" (UniqueName: \"kubernetes.io/projected/8d109a93-d360-400d-a44b-453f14b06575-kube-api-access-tmcmc\") pod \"8d109a93-d360-400d-a44b-453f14b06575\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") "
Nov 03 17:20:28.375116 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.374877    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-tmp-dir\") pod \"8d109a93-d360-400d-a44b-453f14b06575\" (UID: \"8d109a93-d360-400d-a44b-453f14b06575\") "
Nov 03 17:20:28.375116 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.374938    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-model-cache" pod "8d109a93-d360-400d-a44b-453f14b06575" (UID: "8d109a93-d360-400d-a44b-453f14b06575"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:20:28.375116 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.374954    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-home" pod "8d109a93-d360-400d-a44b-453f14b06575" (UID: "8d109a93-d360-400d-a44b-453f14b06575"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:20:28.375299 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.375170    5605 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-home\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:20:28.375299 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.375191    5605 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-model-cache\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:20:28.375299 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.375166    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-tmp-dir" pod "8d109a93-d360-400d-a44b-453f14b06575" (UID: "8d109a93-d360-400d-a44b-453f14b06575"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:20:28.377136 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.377107    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d109a93-d360-400d-a44b-453f14b06575-tls-certs" pod "8d109a93-d360-400d-a44b-453f14b06575" (UID: "8d109a93-d360-400d-a44b-453f14b06575"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:20:28.377242 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.377167    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d109a93-d360-400d-a44b-453f14b06575-kube-api-access-tmcmc" pod "8d109a93-d360-400d-a44b-453f14b06575" (UID: "8d109a93-d360-400d-a44b-453f14b06575"). InnerVolumeSpecName "kube-api-access-tmcmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:20:28.377242 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.377171    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-dshm" pod "8d109a93-d360-400d-a44b-453f14b06575" (UID: "8d109a93-d360-400d-a44b-453f14b06575"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:20:28.476858 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.476827    5605 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d109a93-d360-400d-a44b-453f14b06575-tls-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:20:28.476858 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.476860    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmcmc\" (UniqueName: \"kubernetes.io/projected/8d109a93-d360-400d-a44b-453f14b06575-kube-api-access-tmcmc\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:20:28.477078 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.476897    5605 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-tmp-dir\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:20:28.477078 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:28.476906    5605 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d109a93-d360-400d-a44b-453f14b06575-dshm\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:20:29.027484 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:29.027443    5605 generic.go:356] "Generic (PLEG): container finished" podID="8d109a93-d360-400d-a44b-453f14b06575" containerID="9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586" exitCode=0
Nov 03 17:20:29.027921 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:29.027502    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"
Nov 03 17:20:29.027921 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:29.027531    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5" event={"ID":"8d109a93-d360-400d-a44b-453f14b06575","Type":"ContainerDied","Data":"9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586"}
Nov 03 17:20:29.027921 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:29.027571    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5" event={"ID":"8d109a93-d360-400d-a44b-453f14b06575","Type":"ContainerDied","Data":"42af2373edf94964067443197f92e9cbbd09e72e492894a4f17b091f42d74f7d"}
Nov 03 17:20:29.027921 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:29.027587    5605 scope.go:122] "RemoveContainer" containerID="9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586"
Nov 03 17:20:29.037303 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:29.037288    5605 scope.go:122] "RemoveContainer" containerID="9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586"
Nov 03 17:20:29.037565 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:20:29.037546    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586\": container with ID starting with 9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586 not found: ID does not exist" containerID="9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586"
Nov 03 17:20:29.037608 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:29.037575    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586"} err="failed to get container status \"9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586\": rpc error: code = NotFound desc = could not find container \"9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586\": container with ID starting with 9db7ab8a87b89ce2b13b329455cdfa217e7ea397bd8096571d9ebfe002352586 not found: ID does not exist"
Nov 03 17:20:29.199718 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:29.199688    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"]
Nov 03 17:20:29.250710 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:29.250677    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-dd9d4320-kserve-597744c7c-868x5"]
Nov 03 17:20:30.306897 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:30.306868    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d109a93-d360-400d-a44b-453f14b06575" path="/var/lib/kubelet/pods/8d109a93-d360-400d-a44b-453f14b06575/volumes"
Nov 03 17:20:33.544159 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:20:33.544112    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:20:34.549110 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:34.549055    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:20:34.549631 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:34.549125    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:20:46.335056 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:46.335019    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-zgbzx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:20:46.335692 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:20:46.335659    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-zgbzx" podUID="04b8d456-e273-4b57-94b7-7bffc3831f87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:21:03.391038 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:21:03.390999    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:21:09.337645 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.337608    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"]
Nov 03 17:21:09.338100 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.337972    5605 cpu_manager.go:414] "containerMap: removing container" podUID="8d109a93-d360-400d-a44b-453f14b06575" containerName="main"
Nov 03 17:21:09.338100 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.337983    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d109a93-d360-400d-a44b-453f14b06575" containerName="main"
Nov 03 17:21:09.338100 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.338061    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d109a93-d360-400d-a44b-453f14b06575" containerName="main"
Nov 03 17:21:09.342287 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.342258    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.355022 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.354997    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df72b866-ca01-48e1-8fe7-af64230b518f-maas-api-tls\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.355154 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.355128    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcm5n\" (UniqueName: \"kubernetes.io/projected/df72b866-ca01-48e1-8fe7-af64230b518f-kube-api-access-rcm5n\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.355211 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.355187    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df72b866-ca01-48e1-8fe7-af64230b518f-openshift-service-ca\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.355255 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.355211    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df72b866-ca01-48e1-8fe7-af64230b518f-metrics-certs\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.449063 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.449021    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"]
Nov 03 17:21:09.456183 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.456154    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df72b866-ca01-48e1-8fe7-af64230b518f-metrics-certs\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.456317 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.456236    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df72b866-ca01-48e1-8fe7-af64230b518f-maas-api-tls\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.456365 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.456318    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcm5n\" (UniqueName: \"kubernetes.io/projected/df72b866-ca01-48e1-8fe7-af64230b518f-kube-api-access-rcm5n\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.456365 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.456342    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df72b866-ca01-48e1-8fe7-af64230b518f-openshift-service-ca\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.457104 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.457081    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df72b866-ca01-48e1-8fe7-af64230b518f-openshift-service-ca\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.458956 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.458931    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df72b866-ca01-48e1-8fe7-af64230b518f-maas-api-tls\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.459047 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.458984    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df72b866-ca01-48e1-8fe7-af64230b518f-metrics-certs\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.516479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.516438    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcm5n\" (UniqueName: \"kubernetes.io/projected/df72b866-ca01-48e1-8fe7-af64230b518f-kube-api-access-rcm5n\") pod \"maas-api-855b85cfb4-vhjhp\" (UID: \"df72b866-ca01-48e1-8fe7-af64230b518f\") " pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:09.656266 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:09.656172    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:10.033703 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:10.033575    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"]
Nov 03 17:21:10.036515 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:21:10.036486    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf72b866_ca01_48e1_8fe7_af64230b518f.slice/crio-608fd9ca4b73f5cce94892de7e6b9b2962ed77ed360cb332eeb17dfaf4200d2b WatchSource:0}: Error finding container 608fd9ca4b73f5cce94892de7e6b9b2962ed77ed360cb332eeb17dfaf4200d2b: Status 404 returned error can't find the container with id 608fd9ca4b73f5cce94892de7e6b9b2962ed77ed360cb332eeb17dfaf4200d2b
Nov 03 17:21:10.037842 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:10.037825    5605 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 17:21:10.247261 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:10.247229    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp" event={"ID":"df72b866-ca01-48e1-8fe7-af64230b518f","Type":"ContainerStarted","Data":"608fd9ca4b73f5cce94892de7e6b9b2962ed77ed360cb332eeb17dfaf4200d2b"}
Nov 03 17:21:11.253439 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:11.253339    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp" event={"ID":"df72b866-ca01-48e1-8fe7-af64230b518f","Type":"ContainerStarted","Data":"417f7ae7c9c050fd962dc4dbfe008680e54717df0a8eb02aecc489fda2a55651"}
Nov 03 17:21:11.253826 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:11.253446    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:11.375026 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:11.374975    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp" podStartSLOduration=1.5471163620000001 podStartE2EDuration="2.374961545s" totalImagesPullingTime="827.845183ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:21:11.374054916 +0000 UTC m=+4157.198506187" watchObservedRunningTime="2026-11-03 17:21:11.374961545 +0000 UTC m=+4157.199412816"
Nov 03 17:21:16.094369 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.094332    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"]
Nov 03 17:21:16.097899 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.097881    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.153668 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.153623    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-583a0c6c-openshift-default-dockercfg-k69fm\""
Nov 03 17:21:16.205443 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.205410    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"]
Nov 03 17:21:16.223148 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.223112    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istiod-ca-cert\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.223148 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.223159    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-token\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.223355 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.223239    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-socket\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.223355 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.223308    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-podinfo\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.223438 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.223355    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-credential-socket\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.223500 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.223470    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6xqr\" (UniqueName: \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-kube-api-access-n6xqr\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.223552 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.223523    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-certs\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.223661 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.223643    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-data\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.223746 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.223677    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-envoy\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324284 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324250    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istiod-ca-cert\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324284 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324286    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-token\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324544 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324311    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-socket\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324544 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324346    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-podinfo\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324544 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324392    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-credential-socket\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324544 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324418    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6xqr\" (UniqueName: \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-kube-api-access-n6xqr\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324544 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324439    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-certs\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324544 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324498    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-data\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324544 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324524    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-envoy\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324956 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324798    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-credential-socket\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.324956 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324868    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-certs\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.325063 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.324965    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-data\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.325063 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.325049    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-socket\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.325148 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.325097    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istiod-ca-cert\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.327266 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.327242    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-envoy\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.327683 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.327658    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-podinfo\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.388805 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.386874    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-token\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.388805 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.386918    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6xqr\" (UniqueName: \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-kube-api-access-n6xqr\") pod \"e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") " pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.409714 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.409681    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:16.856057 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.856027    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"]
Nov 03 17:21:16.858118 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:21:16.858096    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72f3a3d_2429_495f_8ffe_36dd74dc72dd.slice/crio-f1933b918ff9593b7e5fab958c6799ac2199c77676a3cda47cc39033eccb5684 WatchSource:0}: Error finding container f1933b918ff9593b7e5fab958c6799ac2199c77676a3cda47cc39033eccb5684: Status 404 returned error can't find the container with id f1933b918ff9593b7e5fab958c6799ac2199c77676a3cda47cc39033eccb5684
Nov 03 17:21:16.860356 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.860318    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:21:16.860455 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.860391    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:21:16.860455 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:16.860417    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:21:17.263667 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:17.263582    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-855b85cfb4-vhjhp"
Nov 03 17:21:17.290050 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:17.290018    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" event={"ID":"b72f3a3d-2429-495f-8ffe-36dd74dc72dd","Type":"ContainerStarted","Data":"80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf"}
Nov 03 17:21:17.290243 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:17.290075    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" event={"ID":"b72f3a3d-2429-495f-8ffe-36dd74dc72dd","Type":"ContainerStarted","Data":"f1933b918ff9593b7e5fab958c6799ac2199c77676a3cda47cc39033eccb5684"}
Nov 03 17:21:17.410382 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:17.410344    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:17.415655 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:17.415625    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:17.423069 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:17.423029    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"]
Nov 03 17:21:17.423310 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:17.423273    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c" podUID="703df78f-e3d3-4876-b5fb-72e268c21839" containerName="maas-api" containerID="cri-o://250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c" gracePeriod=30
Nov 03 17:21:17.521984 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:17.521871    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" podStartSLOduration=1.521857454 podStartE2EDuration="1.521857454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:21:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:21:17.521488148 +0000 UTC m=+4163.345939419" watchObservedRunningTime="2026-11-03 17:21:17.521857454 +0000 UTC m=+4163.346308722"
Nov 03 17:21:17.730250 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:17.730212    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c" podUID="703df78f-e3d3-4876-b5fb-72e268c21839" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.24:8443/health\": dial tcp 10.133.0.24:8443: connect: connection refused"
Nov 03 17:21:18.295526 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.295494    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:18.296487 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.296466    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:18.681031 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.680975    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 17:21:18.753201 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.753162    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/703df78f-e3d3-4876-b5fb-72e268c21839-kube-api-access-xpnlj\" (UniqueName: \"kubernetes.io/projected/703df78f-e3d3-4876-b5fb-72e268c21839-kube-api-access-xpnlj\") pod \"703df78f-e3d3-4876-b5fb-72e268c21839\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") "
Nov 03 17:21:18.753400 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.753269    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-metrics-certs\" (UniqueName: \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-metrics-certs\") pod \"703df78f-e3d3-4876-b5fb-72e268c21839\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") "
Nov 03 17:21:18.753400 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.753297    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-maas-api-tls\") pod \"703df78f-e3d3-4876-b5fb-72e268c21839\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") "
Nov 03 17:21:18.753400 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.753355    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/703df78f-e3d3-4876-b5fb-72e268c21839-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/703df78f-e3d3-4876-b5fb-72e268c21839-openshift-service-ca\") pod \"703df78f-e3d3-4876-b5fb-72e268c21839\" (UID: \"703df78f-e3d3-4876-b5fb-72e268c21839\") "
Nov 03 17:21:18.753771 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.753713    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/703df78f-e3d3-4876-b5fb-72e268c21839-openshift-service-ca" pod "703df78f-e3d3-4876-b5fb-72e268c21839" (UID: "703df78f-e3d3-4876-b5fb-72e268c21839"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:21:18.755602 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.755574    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-metrics-certs" pod "703df78f-e3d3-4876-b5fb-72e268c21839" (UID: "703df78f-e3d3-4876-b5fb-72e268c21839"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:21:18.755694 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.755615    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-maas-api-tls" pod "703df78f-e3d3-4876-b5fb-72e268c21839" (UID: "703df78f-e3d3-4876-b5fb-72e268c21839"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:21:18.755694 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.755628    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/703df78f-e3d3-4876-b5fb-72e268c21839-kube-api-access-xpnlj" pod "703df78f-e3d3-4876-b5fb-72e268c21839" (UID: "703df78f-e3d3-4876-b5fb-72e268c21839"). InnerVolumeSpecName "kube-api-access-xpnlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:21:18.854690 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.854603    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpnlj\" (UniqueName: \"kubernetes.io/projected/703df78f-e3d3-4876-b5fb-72e268c21839-kube-api-access-xpnlj\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:21:18.854690 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.854633    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:21:18.854690 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.854646    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703df78f-e3d3-4876-b5fb-72e268c21839-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:21:18.854690 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:18.854658    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/703df78f-e3d3-4876-b5fb-72e268c21839-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:21:19.301131 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:19.301035    5605 generic.go:356] "Generic (PLEG): container finished" podID="703df78f-e3d3-4876-b5fb-72e268c21839" containerID="250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c" exitCode=0
Nov 03 17:21:19.301131 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:19.301083    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"
Nov 03 17:21:19.301131 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:19.301118    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c" event={"ID":"703df78f-e3d3-4876-b5fb-72e268c21839","Type":"ContainerDied","Data":"250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c"}
Nov 03 17:21:19.301659 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:19.301170    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dfc545664-qnt5c" event={"ID":"703df78f-e3d3-4876-b5fb-72e268c21839","Type":"ContainerDied","Data":"412546c9e636a663d604254504556174c2b52a65d5ec2ee4b5d836f8e75826ac"}
Nov 03 17:21:19.301659 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:19.301212    5605 scope.go:122] "RemoveContainer" containerID="250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c"
Nov 03 17:21:19.311086 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:19.311070    5605 scope.go:122] "RemoveContainer" containerID="250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c"
Nov 03 17:21:19.311365 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:21:19.311348    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c\": container with ID starting with 250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c not found: ID does not exist" containerID="250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c"
Nov 03 17:21:19.311419 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:19.311374    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c"} err="failed to get container status \"250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c\": rpc error: code = NotFound desc = could not find container \"250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c\": container with ID starting with 250dc98d8f29afdf647bad3618618e7bc73eca1155fc2358b29d929329102d1c not found: ID does not exist"
Nov 03 17:21:19.473138 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:19.473103    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"]
Nov 03 17:21:19.524259 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:19.524217    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-dfc545664-qnt5c"]
Nov 03 17:21:20.307610 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:20.307576    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="703df78f-e3d3-4876-b5fb-72e268c21839" path="/var/lib/kubelet/pods/703df78f-e3d3-4876-b5fb-72e268c21839/volumes"
Nov 03 17:21:23.362383 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.362353    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"]
Nov 03 17:21:23.362789 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.362712    5605 cpu_manager.go:414] "containerMap: removing container" podUID="703df78f-e3d3-4876-b5fb-72e268c21839" containerName="maas-api"
Nov 03 17:21:23.362789 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.362750    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="703df78f-e3d3-4876-b5fb-72e268c21839" containerName="maas-api"
Nov 03 17:21:23.362870 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.362814    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="703df78f-e3d3-4876-b5fb-72e268c21839" containerName="maas-api"
Nov 03 17:21:23.367144 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.367127    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:21:23.486612 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.486572    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-583a0c6c\""
Nov 03 17:21:23.486854 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.486637    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-583a0c6c-dockercfg-b74vl\""
Nov 03 17:21:23.488283 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.488261    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"]
Nov 03 17:21:23.497588 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.497566    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"]
Nov 03 17:21:23.501163 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.501145    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:21:23.518934 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.518911    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nh6d\" (UniqueName: \"kubernetes.io/projected/f4c30506-3048-4b84-a253-ffab0a98e7cf-kube-api-access-4nh6d\") pod \"payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f\" (UID: \"f4c30506-3048-4b84-a253-ffab0a98e7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:21:23.519038 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.518967    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4c30506-3048-4b84-a253-ffab0a98e7cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f\" (UID: \"f4c30506-3048-4b84-a253-ffab0a98e7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:21:23.565804 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.565776    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"]
Nov 03 17:21:23.620001 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.619925    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4c30506-3048-4b84-a253-ffab0a98e7cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f\" (UID: \"f4c30506-3048-4b84-a253-ffab0a98e7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:21:23.620143 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.620016    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhtnk\" (UniqueName: \"kubernetes.io/projected/c3787751-81fa-4e2b-8fb7-bdbf8903a988-kube-api-access-vhtnk\") pod \"payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql\" (UID: \"c3787751-81fa-4e2b-8fb7-bdbf8903a988\") " pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:21:23.620143 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.620047    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nh6d\" (UniqueName: \"kubernetes.io/projected/f4c30506-3048-4b84-a253-ffab0a98e7cf-kube-api-access-4nh6d\") pod \"payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f\" (UID: \"f4c30506-3048-4b84-a253-ffab0a98e7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:21:23.620143 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.620075    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3787751-81fa-4e2b-8fb7-bdbf8903a988-plugins-config-volume\") pod \"payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql\" (UID: \"c3787751-81fa-4e2b-8fb7-bdbf8903a988\") " pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:21:23.620611 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.620592    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4c30506-3048-4b84-a253-ffab0a98e7cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f\" (UID: \"f4c30506-3048-4b84-a253-ffab0a98e7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:21:23.682138 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.682107    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nh6d\" (UniqueName: \"kubernetes.io/projected/f4c30506-3048-4b84-a253-ffab0a98e7cf-kube-api-access-4nh6d\") pod \"payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f\" (UID: \"f4c30506-3048-4b84-a253-ffab0a98e7cf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:21:23.721209 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.721181    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhtnk\" (UniqueName: \"kubernetes.io/projected/c3787751-81fa-4e2b-8fb7-bdbf8903a988-kube-api-access-vhtnk\") pod \"payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql\" (UID: \"c3787751-81fa-4e2b-8fb7-bdbf8903a988\") " pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:21:23.721333 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.721228    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3787751-81fa-4e2b-8fb7-bdbf8903a988-plugins-config-volume\") pod \"payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql\" (UID: \"c3787751-81fa-4e2b-8fb7-bdbf8903a988\") " pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:21:23.721842 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.721825    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3787751-81fa-4e2b-8fb7-bdbf8903a988-plugins-config-volume\") pod \"payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql\" (UID: \"c3787751-81fa-4e2b-8fb7-bdbf8903a988\") " pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:21:23.783147 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.783119    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhtnk\" (UniqueName: \"kubernetes.io/projected/c3787751-81fa-4e2b-8fb7-bdbf8903a988-kube-api-access-vhtnk\") pod \"payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql\" (UID: \"c3787751-81fa-4e2b-8fb7-bdbf8903a988\") " pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:21:23.813643 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.813618    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:21:23.979474 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:23.979385    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:21:24.188196 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:24.188173    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"]
Nov 03 17:21:24.190011 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:21:24.189985    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3787751_81fa_4e2b_8fb7_bdbf8903a988.slice/crio-2bc38a9fc74b22f3242cd38bcadcd563cb9bfb78a8e383503f92296ed84f721b WatchSource:0}: Error finding container 2bc38a9fc74b22f3242cd38bcadcd563cb9bfb78a8e383503f92296ed84f721b: Status 404 returned error can't find the container with id 2bc38a9fc74b22f3242cd38bcadcd563cb9bfb78a8e383503f92296ed84f721b
Nov 03 17:21:24.325234 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:24.325200    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql" event={"ID":"c3787751-81fa-4e2b-8fb7-bdbf8903a988","Type":"ContainerStarted","Data":"2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7"}
Nov 03 17:21:24.325416 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:24.325254    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql" event={"ID":"c3787751-81fa-4e2b-8fb7-bdbf8903a988","Type":"ContainerStarted","Data":"2bc38a9fc74b22f3242cd38bcadcd563cb9bfb78a8e383503f92296ed84f721b"}
Nov 03 17:21:24.325416 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:24.325284    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:21:24.356112 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:24.356087    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"]
Nov 03 17:21:24.358669 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:21:24.358644    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c30506_3048_4b84_a253_ffab0a98e7cf.slice/crio-9ec1ea34bef72b24174d65ca9c7fb9743f48bdaadf9bc3d2603bd645ed409885 WatchSource:0}: Error finding container 9ec1ea34bef72b24174d65ca9c7fb9743f48bdaadf9bc3d2603bd645ed409885: Status 404 returned error can't find the container with id 9ec1ea34bef72b24174d65ca9c7fb9743f48bdaadf9bc3d2603bd645ed409885
Nov 03 17:21:24.446720 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:24.446664    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql" podStartSLOduration=1.4466495830000001 podStartE2EDuration="1.446649583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:21:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:21:24.445591302 +0000 UTC m=+4170.270042572" watchObservedRunningTime="2026-11-03 17:21:24.446649583 +0000 UTC m=+4170.271100831"
Nov 03 17:21:25.331915 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.331871    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f" event={"ID":"f4c30506-3048-4b84-a253-ffab0a98e7cf","Type":"ContainerStarted","Data":"151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23"}
Nov 03 17:21:25.332152 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.331944    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f" event={"ID":"f4c30506-3048-4b84-a253-ffab0a98e7cf","Type":"ContainerStarted","Data":"9ec1ea34bef72b24174d65ca9c7fb9743f48bdaadf9bc3d2603bd645ed409885"}
Nov 03 17:21:25.455991 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.455939    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f" podStartSLOduration=2.455926129 podStartE2EDuration="2.455926129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:21:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:21:25.454616817 +0000 UTC m=+4171.279068087" watchObservedRunningTime="2026-11-03 17:21:25.455926129 +0000 UTC m=+4171.280377399"
Nov 03 17:21:25.583922 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.583823    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"]
Nov 03 17:21:25.588065 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.588036    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.644462 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.644433    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-583a0c6c-serving-cert\""
Nov 03 17:21:25.648842 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.648711    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"]
Nov 03 17:21:25.744026 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.743988    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-maas-api-tls\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.744239 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.744102    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0f0945b-6472-4aed-b2e9-45ad87437c68-openshift-service-ca\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.744239 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.744131    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq44h\" (UniqueName: \"kubernetes.io/projected/f0f0945b-6472-4aed-b2e9-45ad87437c68-kube-api-access-nq44h\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.744239 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.744163    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-metrics-certs\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.846071 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.845555    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-maas-api-tls\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.846071 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.845661    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0f0945b-6472-4aed-b2e9-45ad87437c68-openshift-service-ca\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.846071 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.845693    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq44h\" (UniqueName: \"kubernetes.io/projected/f0f0945b-6472-4aed-b2e9-45ad87437c68-kube-api-access-nq44h\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.846071 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.845769    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-metrics-certs\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.857272 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:21:25.847053    5605 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-serving-cert: secret "maas-api-e2e-ait-583a0c6c-serving-cert" not found
Nov 03 17:21:25.857272 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:21:25.847167    5605 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-maas-api-tls podName:f0f0945b-6472-4aed-b2e9-45ad87437c68 nodeName:}" failed. No retries permitted until 2026-11-03 17:21:26.347141909 +0000 UTC m=+4172.171593182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-maas-api-tls") pod "maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh" (UID: "f0f0945b-6472-4aed-b2e9-45ad87437c68") : secret "maas-api-e2e-ait-583a0c6c-serving-cert" not found
Nov 03 17:21:25.857272 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.847292    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0f0945b-6472-4aed-b2e9-45ad87437c68-openshift-service-ca\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.858904 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.858874    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-metrics-certs\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:25.910229 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:25.910202    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq44h\" (UniqueName: \"kubernetes.io/projected/f0f0945b-6472-4aed-b2e9-45ad87437c68-kube-api-access-nq44h\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:26.336901 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:26.336873    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:21:26.351454 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:26.351423    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-maas-api-tls\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:26.354126 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:26.354104    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-maas-api-tls\") pod \"maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:26.501737 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:26.501686    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:26.881259 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:26.881230    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"]
Nov 03 17:21:26.883799 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:21:26.883770    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f0945b_6472_4aed_b2e9_45ad87437c68.slice/crio-e90c9d4f868e5223d2846c9f45b5588d7457d5d9664ae3ab4fd0ea6d31b616ed WatchSource:0}: Error finding container e90c9d4f868e5223d2846c9f45b5588d7457d5d9664ae3ab4fd0ea6d31b616ed: Status 404 returned error can't find the container with id e90c9d4f868e5223d2846c9f45b5588d7457d5d9664ae3ab4fd0ea6d31b616ed
Nov 03 17:21:27.341734 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:27.341701    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh" event={"ID":"f0f0945b-6472-4aed-b2e9-45ad87437c68","Type":"ContainerStarted","Data":"e90c9d4f868e5223d2846c9f45b5588d7457d5d9664ae3ab4fd0ea6d31b616ed"}
Nov 03 17:21:28.347807 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:28.347769    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh" event={"ID":"f0f0945b-6472-4aed-b2e9-45ad87437c68","Type":"ContainerStarted","Data":"4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123"}
Nov 03 17:21:28.348209 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:28.347898    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:28.471170 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:28.471078    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh" podStartSLOduration=2.519780987 podStartE2EDuration="3.471057118s" totalImagesPullingTime="951.276131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:21:28.469864827 +0000 UTC m=+4174.294316097" watchObservedRunningTime="2026-11-03 17:21:28.471057118 +0000 UTC m=+4174.295508383"
Nov 03 17:21:33.390842 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:21:33.390804    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:21:34.358381 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:34.358350    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:21:35.335262 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:35.335237    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:21:37.344956 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:37.344926    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:21:43.443408 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:43.443376    5605 generic.go:356] "Generic (PLEG): container finished" podID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerID="80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf" exitCode=0
Nov 03 17:21:43.443860 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:43.443438    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" event={"ID":"b72f3a3d-2429-495f-8ffe-36dd74dc72dd","Type":"ContainerDied","Data":"80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf"}
Nov 03 17:21:43.443860 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:43.443805    5605 scope.go:122] "RemoveContainer" containerID="80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf"
Nov 03 17:21:43.445199 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:43.445150    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:21:43.445309 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:43.445239    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:21:43.445309 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:43.445274    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:21:44.449652 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:44.449612    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" event={"ID":"b72f3a3d-2429-495f-8ffe-36dd74dc72dd","Type":"ContainerStarted","Data":"eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49"}
Nov 03 17:21:45.410052 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:45.410017    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:45.411422 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:45.411391    5605 patch_prober.go:29] interesting pod/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Nov 03 17:21:45.411587 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:45.411438    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Nov 03 17:21:46.410847 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:46.410808    5605 patch_prober.go:29] interesting pod/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Nov 03 17:21:46.411365 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:46.410883    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Nov 03 17:21:47.410496 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:47.410457    5605 patch_prober.go:29] interesting pod/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Nov 03 17:21:47.410693 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:47.410508    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Nov 03 17:21:48.299765 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:48.298823    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:48.410041 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:48.410006    5605 patch_prober.go:29] interesting pod/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Nov 03 17:21:48.410225 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:48.410058    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Nov 03 17:21:49.410944 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:49.410905    5605 patch_prober.go:29] interesting pod/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Nov 03 17:21:49.411408 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:49.410973    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Nov 03 17:21:50.414124 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:50.414092    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:50.415234 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:50.415217    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:21:54.839661 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:54.839632    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:21:54.857127 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:21:54.857101    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:22:03.392517 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:22:03.392480    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:22:07.531397 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:07.531364    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"]
Nov 03 17:22:07.639490 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:07.639450    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"]
Nov 03 17:22:07.749103 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:07.749070    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"]
Nov 03 17:22:07.749334 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:07.749312    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh" podUID="f0f0945b-6472-4aed-b2e9-45ad87437c68" containerName="maas-api" containerID="cri-o://4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123" gracePeriod=30
Nov 03 17:22:08.004303 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.004275    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:22:08.108247 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.108202    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-maas-api-tls\") pod \"f0f0945b-6472-4aed-b2e9-45ad87437c68\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") "
Nov 03 17:22:08.108453 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.108268    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0f0945b-6472-4aed-b2e9-45ad87437c68-kube-api-access-nq44h\" (UniqueName: \"kubernetes.io/projected/f0f0945b-6472-4aed-b2e9-45ad87437c68-kube-api-access-nq44h\") pod \"f0f0945b-6472-4aed-b2e9-45ad87437c68\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") "
Nov 03 17:22:08.108453 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.108380    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-metrics-certs\") pod \"f0f0945b-6472-4aed-b2e9-45ad87437c68\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") "
Nov 03 17:22:08.108590 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.108539    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0f0945b-6472-4aed-b2e9-45ad87437c68-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0f0945b-6472-4aed-b2e9-45ad87437c68-openshift-service-ca\") pod \"f0f0945b-6472-4aed-b2e9-45ad87437c68\" (UID: \"f0f0945b-6472-4aed-b2e9-45ad87437c68\") "
Nov 03 17:22:08.108991 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.108954    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0f0945b-6472-4aed-b2e9-45ad87437c68-openshift-service-ca" pod "f0f0945b-6472-4aed-b2e9-45ad87437c68" (UID: "f0f0945b-6472-4aed-b2e9-45ad87437c68"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:22:08.110608 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.110584    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-metrics-certs" pod "f0f0945b-6472-4aed-b2e9-45ad87437c68" (UID: "f0f0945b-6472-4aed-b2e9-45ad87437c68"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:22:08.111040 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.111018    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-maas-api-tls" pod "f0f0945b-6472-4aed-b2e9-45ad87437c68" (UID: "f0f0945b-6472-4aed-b2e9-45ad87437c68"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:22:08.111148 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.111052    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0f0945b-6472-4aed-b2e9-45ad87437c68-kube-api-access-nq44h" pod "f0f0945b-6472-4aed-b2e9-45ad87437c68" (UID: "f0f0945b-6472-4aed-b2e9-45ad87437c68"). InnerVolumeSpecName "kube-api-access-nq44h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:22:08.210275 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.210240    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:08.210275 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.210273    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq44h\" (UniqueName: \"kubernetes.io/projected/f0f0945b-6472-4aed-b2e9-45ad87437c68-kube-api-access-nq44h\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:08.210275 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.210284    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0f0945b-6472-4aed-b2e9-45ad87437c68-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:08.210598 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.210294    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0f0945b-6472-4aed-b2e9-45ad87437c68-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:08.563291 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.563211    5605 generic.go:356] "Generic (PLEG): container finished" podID="f0f0945b-6472-4aed-b2e9-45ad87437c68" containerID="4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123" exitCode=0
Nov 03 17:22:08.563291 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.563261    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"
Nov 03 17:22:08.563291 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.563290    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh" event={"ID":"f0f0945b-6472-4aed-b2e9-45ad87437c68","Type":"ContainerDied","Data":"4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123"}
Nov 03 17:22:08.563912 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.563333    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh" event={"ID":"f0f0945b-6472-4aed-b2e9-45ad87437c68","Type":"ContainerDied","Data":"e90c9d4f868e5223d2846c9f45b5588d7457d5d9664ae3ab4fd0ea6d31b616ed"}
Nov 03 17:22:08.563912 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.563360    5605 scope.go:122] "RemoveContainer" containerID="4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123"
Nov 03 17:22:08.572086 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.572067    5605 scope.go:122] "RemoveContainer" containerID="4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123"
Nov 03 17:22:08.572364 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:22:08.572346    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123\": container with ID starting with 4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123 not found: ID does not exist" containerID="4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123"
Nov 03 17:22:08.572433 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.572378    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123"} err="failed to get container status \"4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123\": rpc error: code = NotFound desc = could not find container \"4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123\": container with ID starting with 4003cbaad2625624e8e1d3e4deb338441501641497689094d6aeeda068f0c123 not found: ID does not exist"
Nov 03 17:22:08.741450 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.741411    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"]
Nov 03 17:22:08.743419 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:08.743371    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-583a0c6c-77d774f57d-5w4wh"]
Nov 03 17:22:10.306857 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:10.306824    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0f0945b-6472-4aed-b2e9-45ad87437c68" path="/var/lib/kubelet/pods/f0f0945b-6472-4aed-b2e9-45ad87437c68/volumes"
Nov 03 17:22:12.532150 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.532110    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql" podUID="c3787751-81fa-4e2b-8fb7-bdbf8903a988" containerName="payload-processing" containerID="cri-o://2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7" gracePeriod=25
Nov 03 17:22:12.640399 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.640336    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f" podUID="f4c30506-3048-4b84-a253-ffab0a98e7cf" containerName="payload-pre-processing" containerID="cri-o://151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23" gracePeriod=25
Nov 03 17:22:12.787097 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.787010    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:22:12.856979 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.856950    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3787751-81fa-4e2b-8fb7-bdbf8903a988-kube-api-access-vhtnk\" (UniqueName: \"kubernetes.io/projected/c3787751-81fa-4e2b-8fb7-bdbf8903a988-kube-api-access-vhtnk\") pod \"c3787751-81fa-4e2b-8fb7-bdbf8903a988\" (UID: \"c3787751-81fa-4e2b-8fb7-bdbf8903a988\") "
Nov 03 17:22:12.857154 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.857028    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3787751-81fa-4e2b-8fb7-bdbf8903a988-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3787751-81fa-4e2b-8fb7-bdbf8903a988-plugins-config-volume\") pod \"c3787751-81fa-4e2b-8fb7-bdbf8903a988\" (UID: \"c3787751-81fa-4e2b-8fb7-bdbf8903a988\") "
Nov 03 17:22:12.857429 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.857401    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3787751-81fa-4e2b-8fb7-bdbf8903a988-plugins-config-volume" pod "c3787751-81fa-4e2b-8fb7-bdbf8903a988" (UID: "c3787751-81fa-4e2b-8fb7-bdbf8903a988"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:22:12.859911 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.859882    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3787751-81fa-4e2b-8fb7-bdbf8903a988-kube-api-access-vhtnk" pod "c3787751-81fa-4e2b-8fb7-bdbf8903a988" (UID: "c3787751-81fa-4e2b-8fb7-bdbf8903a988"). InnerVolumeSpecName "kube-api-access-vhtnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:22:12.885762 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.885740    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:22:12.958599 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.958565    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4c30506-3048-4b84-a253-ffab0a98e7cf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4c30506-3048-4b84-a253-ffab0a98e7cf-plugins-config-volume\") pod \"f4c30506-3048-4b84-a253-ffab0a98e7cf\" (UID: \"f4c30506-3048-4b84-a253-ffab0a98e7cf\") "
Nov 03 17:22:12.958599 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.958599    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4c30506-3048-4b84-a253-ffab0a98e7cf-kube-api-access-4nh6d\" (UniqueName: \"kubernetes.io/projected/f4c30506-3048-4b84-a253-ffab0a98e7cf-kube-api-access-4nh6d\") pod \"f4c30506-3048-4b84-a253-ffab0a98e7cf\" (UID: \"f4c30506-3048-4b84-a253-ffab0a98e7cf\") "
Nov 03 17:22:12.958959 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.958941    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhtnk\" (UniqueName: \"kubernetes.io/projected/c3787751-81fa-4e2b-8fb7-bdbf8903a988-kube-api-access-vhtnk\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:12.958959 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.958956    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3787751-81fa-4e2b-8fb7-bdbf8903a988-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:12.959087 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.958963    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4c30506-3048-4b84-a253-ffab0a98e7cf-plugins-config-volume" pod "f4c30506-3048-4b84-a253-ffab0a98e7cf" (UID: "f4c30506-3048-4b84-a253-ffab0a98e7cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:22:12.960998 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:12.960975    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4c30506-3048-4b84-a253-ffab0a98e7cf-kube-api-access-4nh6d" pod "f4c30506-3048-4b84-a253-ffab0a98e7cf" (UID: "f4c30506-3048-4b84-a253-ffab0a98e7cf"). InnerVolumeSpecName "kube-api-access-4nh6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:22:13.060570 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.060486    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4c30506-3048-4b84-a253-ffab0a98e7cf-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:13.060570 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.060516    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nh6d\" (UniqueName: \"kubernetes.io/projected/f4c30506-3048-4b84-a253-ffab0a98e7cf-kube-api-access-4nh6d\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:13.591633 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.591601    5605 generic.go:356] "Generic (PLEG): container finished" podID="c3787751-81fa-4e2b-8fb7-bdbf8903a988" containerID="2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7" exitCode=0
Nov 03 17:22:13.592152 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.591651    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"
Nov 03 17:22:13.592152 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.591686    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql" event={"ID":"c3787751-81fa-4e2b-8fb7-bdbf8903a988","Type":"ContainerDied","Data":"2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7"}
Nov 03 17:22:13.592152 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.591766    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql" event={"ID":"c3787751-81fa-4e2b-8fb7-bdbf8903a988","Type":"ContainerDied","Data":"2bc38a9fc74b22f3242cd38bcadcd563cb9bfb78a8e383503f92296ed84f721b"}
Nov 03 17:22:13.592152 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.591801    5605 scope.go:122] "RemoveContainer" containerID="2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7"
Nov 03 17:22:13.593159 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.593135    5605 generic.go:356] "Generic (PLEG): container finished" podID="f4c30506-3048-4b84-a253-ffab0a98e7cf" containerID="151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23" exitCode=0
Nov 03 17:22:13.593245 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.593165    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f" event={"ID":"f4c30506-3048-4b84-a253-ffab0a98e7cf","Type":"ContainerDied","Data":"151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23"}
Nov 03 17:22:13.593245 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.593222    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f" event={"ID":"f4c30506-3048-4b84-a253-ffab0a98e7cf","Type":"ContainerDied","Data":"9ec1ea34bef72b24174d65ca9c7fb9743f48bdaadf9bc3d2603bd645ed409885"}
Nov 03 17:22:13.593245 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.593177    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"
Nov 03 17:22:13.601343 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.601319    5605 scope.go:122] "RemoveContainer" containerID="2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7"
Nov 03 17:22:13.601604 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:22:13.601586    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7\": container with ID starting with 2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7 not found: ID does not exist" containerID="2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7"
Nov 03 17:22:13.601689 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.601613    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7"} err="failed to get container status \"2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7\": rpc error: code = NotFound desc = could not find container \"2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7\": container with ID starting with 2fb123c67148c4aba2bae5ef6668415b4886c687559bb4ab9b50936e00d355f7 not found: ID does not exist"
Nov 03 17:22:13.601689 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.601628    5605 scope.go:122] "RemoveContainer" containerID="151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23"
Nov 03 17:22:13.609790 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.609770    5605 scope.go:122] "RemoveContainer" containerID="151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23"
Nov 03 17:22:13.610056 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:22:13.610035    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23\": container with ID starting with 151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23 not found: ID does not exist" containerID="151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23"
Nov 03 17:22:13.610106 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.610064    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23"} err="failed to get container status \"151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23\": rpc error: code = NotFound desc = could not find container \"151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23\": container with ID starting with 151128af90786e3d5cda6f65fa98ca28f709ae715893ec186b7bc0e65bfd4d23 not found: ID does not exist"
Nov 03 17:22:13.763448 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.763414    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"]
Nov 03 17:22:13.814709 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.814625    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-583a0c6c-6ffcf84685-smnql"]
Nov 03 17:22:13.929542 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.929503    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"]
Nov 03 17:22:13.937111 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:13.937080    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-583a0c6c-75c4cd9499-rbb5f"]
Nov 03 17:22:14.306944 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:14.306914    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3787751-81fa-4e2b-8fb7-bdbf8903a988" path="/var/lib/kubelet/pods/c3787751-81fa-4e2b-8fb7-bdbf8903a988/volumes"
Nov 03 17:22:14.307342 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:14.307329    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4c30506-3048-4b84-a253-ffab0a98e7cf" path="/var/lib/kubelet/pods/f4c30506-3048-4b84-a253-ffab0a98e7cf/volumes"
Nov 03 17:22:17.108999 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:17.108971    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"]
Nov 03 17:22:17.109430 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:17.109215    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy" containerID="cri-o://eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49" gracePeriod=30
Nov 03 17:22:20.416780 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:20.416742    5605 patch_prober.go:29] interesting pod/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 17:22:20.417274 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:20.416804    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 17:22:22.352920 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.352894    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:22:22.454557 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454467    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-podinfo\") pod \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") "
Nov 03 17:22:22.454557 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454531    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-socket\") pod \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") "
Nov 03 17:22:22.454852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454558    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-data\") pod \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") "
Nov 03 17:22:22.454852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454599    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istiod-ca-cert\") pod \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") "
Nov 03 17:22:22.454852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454624    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-credential-socket\") pod \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") "
Nov 03 17:22:22.454852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454650    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-certs\") pod \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") "
Nov 03 17:22:22.454852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454694    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-envoy\") pod \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") "
Nov 03 17:22:22.454852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454760    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-kube-api-access-n6xqr\" (UniqueName: \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-kube-api-access-n6xqr\") pod \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") "
Nov 03 17:22:22.454852 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454793    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-token\" (UniqueName: \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-token\") pod \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\" (UID: \"b72f3a3d-2429-495f-8ffe-36dd74dc72dd\") "
Nov 03 17:22:22.455240 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454925    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-socket" pod "b72f3a3d-2429-495f-8ffe-36dd74dc72dd" (UID: "b72f3a3d-2429-495f-8ffe-36dd74dc72dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:22:22.455240 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.454968    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-data" pod "b72f3a3d-2429-495f-8ffe-36dd74dc72dd" (UID: "b72f3a3d-2429-495f-8ffe-36dd74dc72dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:22:22.455240 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.455016    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-certs" pod "b72f3a3d-2429-495f-8ffe-36dd74dc72dd" (UID: "b72f3a3d-2429-495f-8ffe-36dd74dc72dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:22:22.455240 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.455207    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istiod-ca-cert" pod "b72f3a3d-2429-495f-8ffe-36dd74dc72dd" (UID: "b72f3a3d-2429-495f-8ffe-36dd74dc72dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:22:22.455440 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.455251    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:22.455440 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.455269    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:22.455440 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.455282    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:22.455440 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.455282    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-credential-socket" pod "b72f3a3d-2429-495f-8ffe-36dd74dc72dd" (UID: "b72f3a3d-2429-495f-8ffe-36dd74dc72dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:22:22.457006 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.456983    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-podinfo" pod "b72f3a3d-2429-495f-8ffe-36dd74dc72dd" (UID: "b72f3a3d-2429-495f-8ffe-36dd74dc72dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 17:22:22.457238 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.457218    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-kube-api-access-n6xqr" pod "b72f3a3d-2429-495f-8ffe-36dd74dc72dd" (UID: "b72f3a3d-2429-495f-8ffe-36dd74dc72dd"). InnerVolumeSpecName "kube-api-access-n6xqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:22:22.457531 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.457513    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-envoy" pod "b72f3a3d-2429-495f-8ffe-36dd74dc72dd" (UID: "b72f3a3d-2429-495f-8ffe-36dd74dc72dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:22:22.457607 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.457585    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-token" pod "b72f3a3d-2429-495f-8ffe-36dd74dc72dd" (UID: "b72f3a3d-2429-495f-8ffe-36dd74dc72dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:22:22.556424 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.556379    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6xqr\" (UniqueName: \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-kube-api-access-n6xqr\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:22.556424 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.556425    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:22.556639 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.556440    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:22.556639 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.556454    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:22.556639 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.556468    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:22.556639 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.556482    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b72f3a3d-2429-495f-8ffe-36dd74dc72dd-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:22:22.637484 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.637444    5605 generic.go:356] "Generic (PLEG): container finished" podID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerID="eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49" exitCode=0
Nov 03 17:22:22.637659 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.637503    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"
Nov 03 17:22:22.637659 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.637528    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" event={"ID":"b72f3a3d-2429-495f-8ffe-36dd74dc72dd","Type":"ContainerDied","Data":"eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49"}
Nov 03 17:22:22.637659 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.637592    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf" event={"ID":"b72f3a3d-2429-495f-8ffe-36dd74dc72dd","Type":"ContainerDied","Data":"f1933b918ff9593b7e5fab958c6799ac2199c77676a3cda47cc39033eccb5684"}
Nov 03 17:22:22.637659 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.637615    5605 scope.go:122] "RemoveContainer" containerID="eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49"
Nov 03 17:22:22.654075 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.654053    5605 scope.go:122] "RemoveContainer" containerID="80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf"
Nov 03 17:22:22.662509 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.662477    5605 scope.go:122] "RemoveContainer" containerID="eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49"
Nov 03 17:22:22.662800 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:22:22.662778    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49\": container with ID starting with eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49 not found: ID does not exist" containerID="eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49"
Nov 03 17:22:22.662850 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.662810    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49"} err="failed to get container status \"eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49\": rpc error: code = NotFound desc = could not find container \"eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49\": container with ID starting with eaeb82ca2a4c5e5417f30d1becbd9664a25a0e00cf52ff92608b327714b09d49 not found: ID does not exist"
Nov 03 17:22:22.662850 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.662829    5605 scope.go:122] "RemoveContainer" containerID="80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf"
Nov 03 17:22:22.663089 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:22:22.663074    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf\": container with ID starting with 80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf not found: ID does not exist" containerID="80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf"
Nov 03 17:22:22.663135 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.663093    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf"} err="failed to get container status \"80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf\": rpc error: code = NotFound desc = could not find container \"80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf\": container with ID starting with 80b710927453048932d11a36b89feb3e5c474ac5390fd7ce71e5163881bae9bf not found: ID does not exist"
Nov 03 17:22:22.808230 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.808145    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"]
Nov 03 17:22:22.814003 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:22.813971    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-583a0c6c-openshift-default-5d94f98679-wchpf"]
Nov 03 17:22:24.306653 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:24.306622    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" path="/var/lib/kubelet/pods/b72f3a3d-2429-495f-8ffe-36dd74dc72dd/volumes"
Nov 03 17:22:28.909624 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.909588    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"]
Nov 03 17:22:28.910032 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.909982    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f0f0945b-6472-4aed-b2e9-45ad87437c68" containerName="maas-api"
Nov 03 17:22:28.910032 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.909995    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0f0945b-6472-4aed-b2e9-45ad87437c68" containerName="maas-api"
Nov 03 17:22:28.910032 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910012    5605 cpu_manager.go:414] "containerMap: removing container" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy"
Nov 03 17:22:28.910032 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910017    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910035    5605 cpu_manager.go:414] "containerMap: removing container" podUID="c3787751-81fa-4e2b-8fb7-bdbf8903a988" containerName="payload-processing"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910041    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3787751-81fa-4e2b-8fb7-bdbf8903a988" containerName="payload-processing"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910048    5605 cpu_manager.go:414] "containerMap: removing container" podUID="f4c30506-3048-4b84-a253-ffab0a98e7cf" containerName="payload-pre-processing"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910053    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4c30506-3048-4b84-a253-ffab0a98e7cf" containerName="payload-pre-processing"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910059    5605 cpu_manager.go:414] "containerMap: removing container" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910064    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910122    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3787751-81fa-4e2b-8fb7-bdbf8903a988" containerName="payload-processing"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910130    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4c30506-3048-4b84-a253-ffab0a98e7cf" containerName="payload-pre-processing"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910136    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910146    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0f0945b-6472-4aed-b2e9-45ad87437c68" containerName="maas-api"
Nov 03 17:22:28.910174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.910153    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="b72f3a3d-2429-495f-8ffe-36dd74dc72dd" containerName="istio-proxy"
Nov 03 17:22:28.912139 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.912123    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:28.981403 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:28.981372    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-52cc05-openshift-default-dockercfg-hndk8\""
Nov 03 17:22:29.017926 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.017891    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-socket\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.018079 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.017940    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-podinfo\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.018079 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.018017    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-envoy\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.018079 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.018049    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5nft\" (UniqueName: \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-kube-api-access-n5nft\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.018235 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.018089    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-certs\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.018235 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.018187    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7da2ab7-15fb-43ef-8602-9426dab2961d-istiod-ca-cert\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.018235 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.018215    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-token\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.018397 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.018245    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-credential-socket\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.018397 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.018267    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-data\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.025971 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.025943    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"]
Nov 03 17:22:29.119213 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.119184    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-socket\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.119373 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.119222    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-podinfo\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.119373 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.119247    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-envoy\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.119373 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.119272    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5nft\" (UniqueName: \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-kube-api-access-n5nft\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.119373 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.119315    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-certs\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.119619 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.119481    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7da2ab7-15fb-43ef-8602-9426dab2961d-istiod-ca-cert\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.119682 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.119610    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-socket\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.119682 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.119652    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-certs\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.120294 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.120260    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7da2ab7-15fb-43ef-8602-9426dab2961d-istiod-ca-cert\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.120428 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.120407    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-token\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.120487 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.120471    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-credential-socket\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.120553 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.120506    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-data\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.120826 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.120805    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-credential-socket\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.120919 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.120899    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-data\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.121617 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.121597    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-envoy\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.121914 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.121897    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-podinfo\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.183670 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.183591    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5nft\" (UniqueName: \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-kube-api-access-n5nft\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.185104 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.185076    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-token\") pod \"e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") " pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.230793 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.230758    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:29.672402 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.672312    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"]
Nov 03 17:22:29.674742 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:22:29.674697    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7da2ab7_15fb_43ef_8602_9426dab2961d.slice/crio-e34824e24b74995034e4b5dac36773e57c1e788de571fcc642c44b6b63d9a4b3 WatchSource:0}: Error finding container e34824e24b74995034e4b5dac36773e57c1e788de571fcc642c44b6b63d9a4b3: Status 404 returned error can't find the container with id e34824e24b74995034e4b5dac36773e57c1e788de571fcc642c44b6b63d9a4b3
Nov 03 17:22:29.676832 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.676798    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:22:29.676930 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.676865    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:22:29.676930 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:29.676897    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:22:30.674279 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:30.674241    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" event={"ID":"b7da2ab7-15fb-43ef-8602-9426dab2961d","Type":"ContainerStarted","Data":"fa5a10b56a6373a3b15e5e4af43f8b567ffa6817dd64438dcefd452b1bac293a"}
Nov 03 17:22:30.674767 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:30.674292    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" event={"ID":"b7da2ab7-15fb-43ef-8602-9426dab2961d","Type":"ContainerStarted","Data":"e34824e24b74995034e4b5dac36773e57c1e788de571fcc642c44b6b63d9a4b3"}
Nov 03 17:22:30.902489 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:30.902422    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" podStartSLOduration=2.902408153 podStartE2EDuration="2.902408153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:22:30.850194932 +0000 UTC m=+4236.674646202" watchObservedRunningTime="2026-11-03 17:22:30.902408153 +0000 UTC m=+4236.726859424"
Nov 03 17:22:31.232408 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:31.232377    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:31.237260 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:31.237236    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:31.679013 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:31.678987    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:31.679953 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:31.679933    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:33.391260 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:22:33.391224    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:22:37.329977 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.329941    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"]
Nov 03 17:22:37.332434 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.332417    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:22:37.424058 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.424024    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd4ss\" (UniqueName: \"kubernetes.io/projected/e17c5aa7-6797-4233-b873-d02961e88ec2-kube-api-access-fd4ss\") pod \"payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr\" (UID: \"e17c5aa7-6797-4233-b873-d02961e88ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:22:37.424058 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.424066    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e17c5aa7-6797-4233-b873-d02961e88ec2-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr\" (UID: \"e17c5aa7-6797-4233-b873-d02961e88ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:22:37.438799 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.438691    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-52cc05\""
Nov 03 17:22:37.439087 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.438706    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-52cc05-dockercfg-2lxkd\""
Nov 03 17:22:37.440399 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.440373    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"]
Nov 03 17:22:37.494938 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.494904    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"]
Nov 03 17:22:37.497533 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.497518    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:22:37.525517 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.525489    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd4ss\" (UniqueName: \"kubernetes.io/projected/e17c5aa7-6797-4233-b873-d02961e88ec2-kube-api-access-fd4ss\") pod \"payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr\" (UID: \"e17c5aa7-6797-4233-b873-d02961e88ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:22:37.525663 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.525527    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e17c5aa7-6797-4233-b873-d02961e88ec2-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr\" (UID: \"e17c5aa7-6797-4233-b873-d02961e88ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:22:37.526294 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.526274    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e17c5aa7-6797-4233-b873-d02961e88ec2-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr\" (UID: \"e17c5aa7-6797-4233-b873-d02961e88ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:22:37.557867 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.557837    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"]
Nov 03 17:22:37.586270 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.586242    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd4ss\" (UniqueName: \"kubernetes.io/projected/e17c5aa7-6797-4233-b873-d02961e88ec2-kube-api-access-fd4ss\") pod \"payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr\" (UID: \"e17c5aa7-6797-4233-b873-d02961e88ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:22:37.627169 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.627135    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6585a454-5a80-446a-98ae-144b68603d71-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr\" (UID: \"6585a454-5a80-446a-98ae-144b68603d71\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:22:37.627341 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.627195    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6nvb\" (UniqueName: \"kubernetes.io/projected/6585a454-5a80-446a-98ae-144b68603d71-kube-api-access-n6nvb\") pod \"payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr\" (UID: \"6585a454-5a80-446a-98ae-144b68603d71\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:22:37.644971 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.644946    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:22:37.728376 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.728343    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6585a454-5a80-446a-98ae-144b68603d71-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr\" (UID: \"6585a454-5a80-446a-98ae-144b68603d71\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:22:37.728524 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.728405    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6nvb\" (UniqueName: \"kubernetes.io/projected/6585a454-5a80-446a-98ae-144b68603d71-kube-api-access-n6nvb\") pod \"payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr\" (UID: \"6585a454-5a80-446a-98ae-144b68603d71\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:22:37.729246 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.729223    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6585a454-5a80-446a-98ae-144b68603d71-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr\" (UID: \"6585a454-5a80-446a-98ae-144b68603d71\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:22:37.789451 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.789416    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6nvb\" (UniqueName: \"kubernetes.io/projected/6585a454-5a80-446a-98ae-144b68603d71-kube-api-access-n6nvb\") pod \"payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr\" (UID: \"6585a454-5a80-446a-98ae-144b68603d71\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:22:37.809311 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:37.809283    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:22:38.020000 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:38.019974    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"]
Nov 03 17:22:38.022697 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:22:38.022666    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode17c5aa7_6797_4233_b873_d02961e88ec2.slice/crio-1ea366918c4d4fe48184c93c04c33fae4c2e18518cf1b6f22f7cff1337ab4aca WatchSource:0}: Error finding container 1ea366918c4d4fe48184c93c04c33fae4c2e18518cf1b6f22f7cff1337ab4aca: Status 404 returned error can't find the container with id 1ea366918c4d4fe48184c93c04c33fae4c2e18518cf1b6f22f7cff1337ab4aca
Nov 03 17:22:38.184871 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:38.184841    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"]
Nov 03 17:22:38.186651 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:22:38.186622    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6585a454_5a80_446a_98ae_144b68603d71.slice/crio-c46858f5f3580b7a6e0b69a3f4771425ab61c3b00b26c34ace888df79f94d6dd WatchSource:0}: Error finding container c46858f5f3580b7a6e0b69a3f4771425ab61c3b00b26c34ace888df79f94d6dd: Status 404 returned error can't find the container with id c46858f5f3580b7a6e0b69a3f4771425ab61c3b00b26c34ace888df79f94d6dd
Nov 03 17:22:38.714100 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:38.714061    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr" event={"ID":"6585a454-5a80-446a-98ae-144b68603d71","Type":"ContainerStarted","Data":"d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af"}
Nov 03 17:22:38.714554 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:38.714130    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr" event={"ID":"6585a454-5a80-446a-98ae-144b68603d71","Type":"ContainerStarted","Data":"c46858f5f3580b7a6e0b69a3f4771425ab61c3b00b26c34ace888df79f94d6dd"}
Nov 03 17:22:38.714554 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:38.714160    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:22:38.715618 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:38.715595    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr" event={"ID":"e17c5aa7-6797-4233-b873-d02961e88ec2","Type":"ContainerStarted","Data":"27b0667c8edb92ea197c98fa2ef75eda70011f319d0e14e584bc7478c89e8bcd"}
Nov 03 17:22:38.715789 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:38.715644    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr" event={"ID":"e17c5aa7-6797-4233-b873-d02961e88ec2","Type":"ContainerStarted","Data":"1ea366918c4d4fe48184c93c04c33fae4c2e18518cf1b6f22f7cff1337ab4aca"}
Nov 03 17:22:38.715789 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:38.715674    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:22:38.838877 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:38.838827    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr" podStartSLOduration=1.838813467 podStartE2EDuration="1.838813467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:22:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:22:38.8384896 +0000 UTC m=+4244.662940882" watchObservedRunningTime="2026-11-03 17:22:38.838813467 +0000 UTC m=+4244.663264735"
Nov 03 17:22:38.975237 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:38.975113    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr" podStartSLOduration=1.9750903480000002 podStartE2EDuration="1.975090348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:22:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:22:38.972800063 +0000 UTC m=+4244.797251335" watchObservedRunningTime="2026-11-03 17:22:38.975090348 +0000 UTC m=+4244.799541620"
Nov 03 17:22:40.218484 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.218453    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"]
Nov 03 17:22:40.221204 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.221184    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.276495 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.276458    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-52cc05-serving-cert\""
Nov 03 17:22:40.328203 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.328175    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"]
Nov 03 17:22:40.353659 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.353624    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.353864 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.353755    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9txvs\" (UniqueName: \"kubernetes.io/projected/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-kube-api-access-9txvs\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.353864 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.353831    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-metrics-certs\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.353945 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.353865    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.454374 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.454344    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9txvs\" (UniqueName: \"kubernetes.io/projected/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-kube-api-access-9txvs\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.454561 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.454412    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-metrics-certs\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.454561 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.454451    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.454640 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.454585    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.455361 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.455253    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.457082 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.457058    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.457195 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.457170    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-metrics-certs\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.514919 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.514847    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9txvs\" (UniqueName: \"kubernetes.io/projected/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-kube-api-access-9txvs\") pod \"maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.534105 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.534072    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:40.942984 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:40.942847    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"]
Nov 03 17:22:40.945719 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:22:40.945691    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94685fd9_3c3c_4bdd_9ec2_b0976c2339aa.slice/crio-34715be4cea98da4495a93f17de717628ba0ebc632ebe5e25a19188f6e4f568e WatchSource:0}: Error finding container 34715be4cea98da4495a93f17de717628ba0ebc632ebe5e25a19188f6e4f568e: Status 404 returned error can't find the container with id 34715be4cea98da4495a93f17de717628ba0ebc632ebe5e25a19188f6e4f568e
Nov 03 17:22:41.731257 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:41.731221    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77" event={"ID":"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa","Type":"ContainerStarted","Data":"34715be4cea98da4495a93f17de717628ba0ebc632ebe5e25a19188f6e4f568e"}
Nov 03 17:22:42.737197 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:42.737156    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77" event={"ID":"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa","Type":"ContainerStarted","Data":"4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51"}
Nov 03 17:22:42.737652 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:42.737273    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:42.859409 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:42.859353    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77" podStartSLOduration=1.9933388 podStartE2EDuration="2.859340405s" totalImagesPullingTime="866.001605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:22:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:22:42.857573498 +0000 UTC m=+4248.682024774" watchObservedRunningTime="2026-11-03 17:22:42.859340405 +0000 UTC m=+4248.683791676"
Nov 03 17:22:48.747485 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:48.747453    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:22:49.723878 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:49.723854    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:22:49.724073 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:49.723922    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:22:57.835421 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:57.835387    5605 generic.go:356] "Generic (PLEG): container finished" podID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerID="fa5a10b56a6373a3b15e5e4af43f8b567ffa6817dd64438dcefd452b1bac293a" exitCode=0
Nov 03 17:22:57.835879 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:57.835469    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" event={"ID":"b7da2ab7-15fb-43ef-8602-9426dab2961d","Type":"ContainerDied","Data":"fa5a10b56a6373a3b15e5e4af43f8b567ffa6817dd64438dcefd452b1bac293a"}
Nov 03 17:22:57.835927 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:57.835897    5605 scope.go:122] "RemoveContainer" containerID="fa5a10b56a6373a3b15e5e4af43f8b567ffa6817dd64438dcefd452b1bac293a"
Nov 03 17:22:57.837220 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:57.837186    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:22:57.837301 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:57.837271    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:22:57.837348 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:57.837312    5605 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892016Ki","pods":"250"}
Nov 03 17:22:58.841256 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:58.841180    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" event={"ID":"b7da2ab7-15fb-43ef-8602-9426dab2961d","Type":"ContainerStarted","Data":"fcf4f9256e2ef6f7a80c39e758b893f6658d9b1a78e6396fea1fdf46b46b60c9"}
Nov 03 17:22:59.231821 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:59.231710    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:22:59.232911 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:59.232883    5605 patch_prober.go:29] interesting pod/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Nov 03 17:22:59.233037 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:22:59.232935    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Nov 03 17:23:00.232776 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:00.232714    5605 patch_prober.go:29] interesting pod/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Nov 03 17:23:00.233268 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:00.232802    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Nov 03 17:23:01.231799 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:01.231763    5605 patch_prober.go:29] interesting pod/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Nov 03 17:23:01.231989 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:01.231814    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Nov 03 17:23:01.679069 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:01.679033    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:23:02.232189 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:02.232151    5605 patch_prober.go:29] interesting pod/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Nov 03 17:23:02.232374 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:02.232212    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Nov 03 17:23:03.232812 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:03.232759    5605 patch_prober.go:29] interesting pod/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Nov 03 17:23:03.233233 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:03.232826    5605 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Nov 03 17:23:03.415876 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:23:03.415839    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:23:04.235963 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:04.235933    5605 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:23:04.236902 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:23:04.236885    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:23:33.391808 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:23:33.391768    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:24:03.391557 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:24:03.391514    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:24:32.974722 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:32.974635    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"]
Nov 03 17:24:33.082425 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:33.082392    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"]
Nov 03 17:24:33.406650 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:24:33.406611    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:24:37.975387 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:37.975337    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr" podUID="6585a454-5a80-446a-98ae-144b68603d71" containerName="payload-processing" containerID="cri-o://d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af" gracePeriod=25
Nov 03 17:24:38.083418 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.083378    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr" podUID="e17c5aa7-6797-4233-b873-d02961e88ec2" containerName="payload-pre-processing" containerID="cri-o://27b0667c8edb92ea197c98fa2ef75eda70011f319d0e14e584bc7478c89e8bcd" gracePeriod=25
Nov 03 17:24:38.231659 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.231591    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:24:38.305437 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.305399    5605 generic.go:356] "Generic (PLEG): container finished" podID="6585a454-5a80-446a-98ae-144b68603d71" containerID="d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af" exitCode=0
Nov 03 17:24:38.305624 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.305509    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"
Nov 03 17:24:38.307280 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.307255    5605 generic.go:356] "Generic (PLEG): container finished" podID="e17c5aa7-6797-4233-b873-d02961e88ec2" containerID="27b0667c8edb92ea197c98fa2ef75eda70011f319d0e14e584bc7478c89e8bcd" exitCode=0
Nov 03 17:24:38.309316 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.309294    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr" event={"ID":"6585a454-5a80-446a-98ae-144b68603d71","Type":"ContainerDied","Data":"d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af"}
Nov 03 17:24:38.309381 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.309356    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr" event={"ID":"6585a454-5a80-446a-98ae-144b68603d71","Type":"ContainerDied","Data":"c46858f5f3580b7a6e0b69a3f4771425ab61c3b00b26c34ace888df79f94d6dd"}
Nov 03 17:24:38.309421 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.309411    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr" event={"ID":"e17c5aa7-6797-4233-b873-d02961e88ec2","Type":"ContainerDied","Data":"27b0667c8edb92ea197c98fa2ef75eda70011f319d0e14e584bc7478c89e8bcd"}
Nov 03 17:24:38.309461 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.309432    5605 scope.go:122] "RemoveContainer" containerID="d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af"
Nov 03 17:24:38.317914 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.317889    5605 scope.go:122] "RemoveContainer" containerID="d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af"
Nov 03 17:24:38.318200 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:24:38.318168    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af\": container with ID starting with d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af not found: ID does not exist" containerID="d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af"
Nov 03 17:24:38.318274 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.318197    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af"} err="failed to get container status \"d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af\": rpc error: code = NotFound desc = could not find container \"d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af\": container with ID starting with d8d4379b9b75e4efe5118517c8d246d123444fabe074b5997b747bef7b0b73af not found: ID does not exist"
Nov 03 17:24:38.351780 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.351760    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:24:38.400531 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.400502    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6585a454-5a80-446a-98ae-144b68603d71-kube-api-access-n6nvb\" (UniqueName: \"kubernetes.io/projected/6585a454-5a80-446a-98ae-144b68603d71-kube-api-access-n6nvb\") pod \"6585a454-5a80-446a-98ae-144b68603d71\" (UID: \"6585a454-5a80-446a-98ae-144b68603d71\") "
Nov 03 17:24:38.400709 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.400546    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6585a454-5a80-446a-98ae-144b68603d71-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6585a454-5a80-446a-98ae-144b68603d71-plugins-config-volume\") pod \"6585a454-5a80-446a-98ae-144b68603d71\" (UID: \"6585a454-5a80-446a-98ae-144b68603d71\") "
Nov 03 17:24:38.400995 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.400971    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6585a454-5a80-446a-98ae-144b68603d71-plugins-config-volume" pod "6585a454-5a80-446a-98ae-144b68603d71" (UID: "6585a454-5a80-446a-98ae-144b68603d71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:24:38.402764 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.402745    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6585a454-5a80-446a-98ae-144b68603d71-kube-api-access-n6nvb" pod "6585a454-5a80-446a-98ae-144b68603d71" (UID: "6585a454-5a80-446a-98ae-144b68603d71"). InnerVolumeSpecName "kube-api-access-n6nvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:24:38.502030 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.501934    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e17c5aa7-6797-4233-b873-d02961e88ec2-kube-api-access-fd4ss\" (UniqueName: \"kubernetes.io/projected/e17c5aa7-6797-4233-b873-d02961e88ec2-kube-api-access-fd4ss\") pod \"e17c5aa7-6797-4233-b873-d02961e88ec2\" (UID: \"e17c5aa7-6797-4233-b873-d02961e88ec2\") "
Nov 03 17:24:38.502030 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.501990    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e17c5aa7-6797-4233-b873-d02961e88ec2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e17c5aa7-6797-4233-b873-d02961e88ec2-plugins-config-volume\") pod \"e17c5aa7-6797-4233-b873-d02961e88ec2\" (UID: \"e17c5aa7-6797-4233-b873-d02961e88ec2\") "
Nov 03 17:24:38.502372 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.502347    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6nvb\" (UniqueName: \"kubernetes.io/projected/6585a454-5a80-446a-98ae-144b68603d71-kube-api-access-n6nvb\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:38.502463 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.502377    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6585a454-5a80-446a-98ae-144b68603d71-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:38.502463 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.502437    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e17c5aa7-6797-4233-b873-d02961e88ec2-plugins-config-volume" pod "e17c5aa7-6797-4233-b873-d02961e88ec2" (UID: "e17c5aa7-6797-4233-b873-d02961e88ec2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:24:38.504371 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.504346    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e17c5aa7-6797-4233-b873-d02961e88ec2-kube-api-access-fd4ss" pod "e17c5aa7-6797-4233-b873-d02961e88ec2" (UID: "e17c5aa7-6797-4233-b873-d02961e88ec2"). InnerVolumeSpecName "kube-api-access-fd4ss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:24:38.603524 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.603487    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd4ss\" (UniqueName: \"kubernetes.io/projected/e17c5aa7-6797-4233-b873-d02961e88ec2-kube-api-access-fd4ss\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:38.603524 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.603521    5605 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e17c5aa7-6797-4233-b873-d02961e88ec2-plugins-config-volume\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:38.732754 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.732699    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"]
Nov 03 17:24:38.739772 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:38.739745    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-52cc05-78fcdf7659-mc2hr"]
Nov 03 17:24:39.314761 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.314658    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr" event={"ID":"e17c5aa7-6797-4233-b873-d02961e88ec2","Type":"ContainerDied","Data":"1ea366918c4d4fe48184c93c04c33fae4c2e18518cf1b6f22f7cff1337ab4aca"}
Nov 03 17:24:39.314761 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.314682    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"
Nov 03 17:24:39.314761 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.314756    5605 scope.go:122] "RemoveContainer" containerID="27b0667c8edb92ea197c98fa2ef75eda70011f319d0e14e584bc7478c89e8bcd"
Nov 03 17:24:39.422745 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.422691    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"]
Nov 03 17:24:39.422996 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.422967    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77" podUID="94685fd9-3c3c-4bdd-9ec2-b0976c2339aa" containerName="maas-api" containerID="cri-o://4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51" gracePeriod=30
Nov 03 17:24:39.486958 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.486923    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"]
Nov 03 17:24:39.492108 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.492083    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-52cc05-7d4ff6f795-qsjgr"]
Nov 03 17:24:39.697551 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.697525    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:24:39.817300 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.817262    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-metrics-certs\" (UniqueName: \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-metrics-certs\") pod \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") "
Nov 03 17:24:39.817499 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.817409    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-openshift-service-ca\") pod \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") "
Nov 03 17:24:39.817499 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.817483    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-kube-api-access-9txvs\" (UniqueName: \"kubernetes.io/projected/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-kube-api-access-9txvs\") pod \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") "
Nov 03 17:24:39.817644 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.817612    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-maas-api-tls\") pod \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\" (UID: \"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa\") "
Nov 03 17:24:39.817812 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.817785    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-openshift-service-ca" pod "94685fd9-3c3c-4bdd-9ec2-b0976c2339aa" (UID: "94685fd9-3c3c-4bdd-9ec2-b0976c2339aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:24:39.818052 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.818033    5605 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-openshift-service-ca\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:39.819580 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.819558    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-metrics-certs" pod "94685fd9-3c3c-4bdd-9ec2-b0976c2339aa" (UID: "94685fd9-3c3c-4bdd-9ec2-b0976c2339aa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:24:39.819669 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.819656    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-kube-api-access-9txvs" pod "94685fd9-3c3c-4bdd-9ec2-b0976c2339aa" (UID: "94685fd9-3c3c-4bdd-9ec2-b0976c2339aa"). InnerVolumeSpecName "kube-api-access-9txvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:24:39.819838 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.819823    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-maas-api-tls" pod "94685fd9-3c3c-4bdd-9ec2-b0976c2339aa" (UID: "94685fd9-3c3c-4bdd-9ec2-b0976c2339aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 17:24:39.919272 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.919176    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9txvs\" (UniqueName: \"kubernetes.io/projected/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-kube-api-access-9txvs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:39.919272 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.919208    5605 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-maas-api-tls\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:39.919272 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:39.919220    5605 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa-metrics-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:40.309829 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.309712    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6585a454-5a80-446a-98ae-144b68603d71" path="/var/lib/kubelet/pods/6585a454-5a80-446a-98ae-144b68603d71/volumes"
Nov 03 17:24:40.310377 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.310347    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e17c5aa7-6797-4233-b873-d02961e88ec2" path="/var/lib/kubelet/pods/e17c5aa7-6797-4233-b873-d02961e88ec2/volumes"
Nov 03 17:24:40.320791 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.320678    5605 generic.go:356] "Generic (PLEG): container finished" podID="94685fd9-3c3c-4bdd-9ec2-b0976c2339aa" containerID="4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51" exitCode=0
Nov 03 17:24:40.320791 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.320766    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"
Nov 03 17:24:40.321227 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.320799    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77" event={"ID":"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa","Type":"ContainerDied","Data":"4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51"}
Nov 03 17:24:40.321227 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.320846    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77" event={"ID":"94685fd9-3c3c-4bdd-9ec2-b0976c2339aa","Type":"ContainerDied","Data":"34715be4cea98da4495a93f17de717628ba0ebc632ebe5e25a19188f6e4f568e"}
Nov 03 17:24:40.321227 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.320875    5605 scope.go:122] "RemoveContainer" containerID="4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51"
Nov 03 17:24:40.350527 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.350495    5605 scope.go:122] "RemoveContainer" containerID="4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51"
Nov 03 17:24:40.351260 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:24:40.351182    5605 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51\": container with ID starting with 4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51 not found: ID does not exist" containerID="4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51"
Nov 03 17:24:40.351260 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.351222    5605 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51"} err="failed to get container status \"4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51\": rpc error: code = NotFound desc = could not find container \"4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51\": container with ID starting with 4ddb1b58892f2d578497f2618d5fc00389249661967f880e0729597cb9cb5b51 not found: ID does not exist"
Nov 03 17:24:40.495222 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.495190    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"]
Nov 03 17:24:40.546191 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:40.546156    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-52cc05-69487fb94-62k77"]
Nov 03 17:24:42.307496 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:42.307461    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94685fd9-3c3c-4bdd-9ec2-b0976c2339aa" path="/var/lib/kubelet/pods/94685fd9-3c3c-4bdd-9ec2-b0976c2339aa/volumes"
Nov 03 17:24:49.185035 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:49.184997    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"]
Nov 03 17:24:49.185572 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:49.185271    5605 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy" containerID="cri-o://fcf4f9256e2ef6f7a80c39e758b893f6658d9b1a78e6396fea1fdf46b46b60c9" gracePeriod=30
Nov 03 17:24:49.259436 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:49.258936    5605 patch_prober.go:29] interesting pod/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 17:24:49.259436 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:49.259019    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 17:24:54.400149 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.400114    5605 generic.go:356] "Generic (PLEG): container finished" podID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerID="fcf4f9256e2ef6f7a80c39e758b893f6658d9b1a78e6396fea1fdf46b46b60c9" exitCode=0
Nov 03 17:24:54.400522 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.400194    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" event={"ID":"b7da2ab7-15fb-43ef-8602-9426dab2961d","Type":"ContainerDied","Data":"fcf4f9256e2ef6f7a80c39e758b893f6658d9b1a78e6396fea1fdf46b46b60c9"}
Nov 03 17:24:54.400522 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.400306    5605 scope.go:122] "RemoveContainer" containerID="fa5a10b56a6373a3b15e5e4af43f8b567ffa6817dd64438dcefd452b1bac293a"
Nov 03 17:24:54.438964 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.438935    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:24:54.467134 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467056    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-credential-socket\") pod \"b7da2ab7-15fb-43ef-8602-9426dab2961d\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") "
Nov 03 17:24:54.467283 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467147    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-envoy\") pod \"b7da2ab7-15fb-43ef-8602-9426dab2961d\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") "
Nov 03 17:24:54.467283 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467189    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7da2ab7-15fb-43ef-8602-9426dab2961d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7da2ab7-15fb-43ef-8602-9426dab2961d-istiod-ca-cert\") pod \"b7da2ab7-15fb-43ef-8602-9426dab2961d\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") "
Nov 03 17:24:54.467283 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467215    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-data\") pod \"b7da2ab7-15fb-43ef-8602-9426dab2961d\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") "
Nov 03 17:24:54.467283 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467256    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-podinfo\") pod \"b7da2ab7-15fb-43ef-8602-9426dab2961d\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") "
Nov 03 17:24:54.467523 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467315    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-kube-api-access-n5nft\" (UniqueName: \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-kube-api-access-n5nft\") pod \"b7da2ab7-15fb-43ef-8602-9426dab2961d\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") "
Nov 03 17:24:54.467523 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467339    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-credential-socket" pod "b7da2ab7-15fb-43ef-8602-9426dab2961d" (UID: "b7da2ab7-15fb-43ef-8602-9426dab2961d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:24:54.467523 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467396    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-token\" (UniqueName: \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-token\") pod \"b7da2ab7-15fb-43ef-8602-9426dab2961d\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") "
Nov 03 17:24:54.467523 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467429    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-socket\") pod \"b7da2ab7-15fb-43ef-8602-9426dab2961d\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") "
Nov 03 17:24:54.467523 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467456    5605 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-certs\") pod \"b7da2ab7-15fb-43ef-8602-9426dab2961d\" (UID: \"b7da2ab7-15fb-43ef-8602-9426dab2961d\") "
Nov 03 17:24:54.467844 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467557    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-data" pod "b7da2ab7-15fb-43ef-8602-9426dab2961d" (UID: "b7da2ab7-15fb-43ef-8602-9426dab2961d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:24:54.467844 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467565    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7da2ab7-15fb-43ef-8602-9426dab2961d-istiod-ca-cert" pod "b7da2ab7-15fb-43ef-8602-9426dab2961d" (UID: "b7da2ab7-15fb-43ef-8602-9426dab2961d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 17:24:54.467844 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467803    5605 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7da2ab7-15fb-43ef-8602-9426dab2961d-istiod-ca-cert\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:54.467844 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467831    5605 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-data\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:54.468234 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467853    5605 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-credential-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:54.468234 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.467861    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-certs" pod "b7da2ab7-15fb-43ef-8602-9426dab2961d" (UID: "b7da2ab7-15fb-43ef-8602-9426dab2961d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:24:54.468234 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.468113    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-socket" pod "b7da2ab7-15fb-43ef-8602-9426dab2961d" (UID: "b7da2ab7-15fb-43ef-8602-9426dab2961d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:24:54.469597 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.469569    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-podinfo" pod "b7da2ab7-15fb-43ef-8602-9426dab2961d" (UID: "b7da2ab7-15fb-43ef-8602-9426dab2961d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 17:24:54.469698 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.469567    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-envoy" pod "b7da2ab7-15fb-43ef-8602-9426dab2961d" (UID: "b7da2ab7-15fb-43ef-8602-9426dab2961d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 17:24:54.469698 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.469642    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-token" pod "b7da2ab7-15fb-43ef-8602-9426dab2961d" (UID: "b7da2ab7-15fb-43ef-8602-9426dab2961d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:24:54.469698 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.469682    5605 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-kube-api-access-n5nft" pod "b7da2ab7-15fb-43ef-8602-9426dab2961d" (UID: "b7da2ab7-15fb-43ef-8602-9426dab2961d"). InnerVolumeSpecName "kube-api-access-n5nft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 17:24:54.569224 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.569182    5605 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5nft\" (UniqueName: \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-kube-api-access-n5nft\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:54.569224 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.569217    5605 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-token\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:54.569224 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.569231    5605 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-socket\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:54.569472 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.569243    5605 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-workload-certs\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:54.569472 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.569254    5605 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-envoy\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:54.569472 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:54.569266    5605 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7da2ab7-15fb-43ef-8602-9426dab2961d-istio-podinfo\") on node \"ip-10-0-142-22.us-west-2.compute.internal\" DevicePath \"\""
Nov 03 17:24:55.407097 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:55.407050    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp" event={"ID":"b7da2ab7-15fb-43ef-8602-9426dab2961d","Type":"ContainerDied","Data":"e34824e24b74995034e4b5dac36773e57c1e788de571fcc642c44b6b63d9a4b3"}
Nov 03 17:24:55.407097 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:55.407087    5605 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"
Nov 03 17:24:55.407604 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:55.407121    5605 scope.go:122] "RemoveContainer" containerID="fcf4f9256e2ef6f7a80c39e758b893f6658d9b1a78e6396fea1fdf46b46b60c9"
Nov 03 17:24:55.630305 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:55.630270    5605 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"]
Nov 03 17:24:55.633492 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:55.633474    5605 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-52cc05-openshift-default-85444f7dcc-7s9hp"]
Nov 03 17:24:56.308593 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:24:56.308554    5605 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" path="/var/lib/kubelet/pods/b7da2ab7-15fb-43ef-8602-9426dab2961d/volumes"
Nov 03 17:25:03.391941 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:25:03.391893    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:25:11.127891 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:25:11.127846    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rn227 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:25:11.129169 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:25:11.128655    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rn227" podUID="b0246a86-f897-42f0-88eb-38c00c8b1bc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:25:33.391940 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:25:33.391893    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:26:03.525791 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:26:03.520281    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:26:04.555926 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:04.555883    5605 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-rj7rb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 17:26:04.556356 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:04.555955    5605 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-rj7rb" podUID="5c54230f-c0e9-4672-b919-e6bd3383c411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 17:26:25.662812 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:25.662779    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-855b85cfb4-vhjhp_df72b866-ca01-48e1-8fe7-af64230b518f/maas-api/0.log"
Nov 03 17:26:33.390543 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:26:33.390506    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:26:41.039960 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:41.039932    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-xj658_b49dafeb-4657-4a77-bcd0-ede25c02a57c/discovery/0.log"
Nov 03 17:26:41.633867 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:41.633841    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fbcdb5d6b-z4mv5_3b41055d-d1c9-4ba3-b938-42c94b091764/kube-auth-proxy/0.log"
Nov 03 17:26:42.612205 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:42.612172    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-gjzpm_dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da/istio-proxy/0.log"
Nov 03 17:26:43.407189 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:43.407153    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-rj7rb_5c54230f-c0e9-4672-b919-e6bd3383c411/istio-proxy/0.log"
Nov 03 17:26:43.681873 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:43.681798    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-rn227_b0246a86-f897-42f0-88eb-38c00c8b1bc3/istio-proxy/0.log"
Nov 03 17:26:44.231952 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:44.231925    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-zgbzx_04b8d456-e273-4b57-94b7-7bffc3831f87/istio-proxy/0.log"
Nov 03 17:26:46.273720 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:46.273690    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-t7rp9_d5cc21a5-ffff-470e-9f8e-41acc80f2132/main/0.log"
Nov 03 17:26:46.556562 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:46.556476    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-mf979_983fedf8-220b-4e70-bf19-1e72aa4391e5/main/0.log"
Nov 03 17:26:47.258105 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:47.258068    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-4jdmt_843c64aa-38ea-496a-bd0c-13158d516c7c/main/0.log"
Nov 03 17:26:47.756737 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:47.756695    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld_f237b2f1-9a28-419e-805a-0d77cbda7230/main/0.log"
Nov 03 17:26:54.866471 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:54.866439    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:26:54.885829 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:26:54.885807    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgfrq_1c560131-5ae7-42ef-975b-23af6909fb89/kube-multus/0.log"
Nov 03 17:27:03.391458 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:27:03.391412    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:27:33.391128 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:27:33.391036    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:27:40.846698 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:27:40.846667    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-855b85cfb4-vhjhp_df72b866-ca01-48e1-8fe7-af64230b518f/maas-api/0.log"
Nov 03 17:27:45.491880 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:27:45.491850    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-xj658_b49dafeb-4657-4a77-bcd0-ede25c02a57c/discovery/0.log"
Nov 03 17:27:46.080648 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:27:46.080617    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fbcdb5d6b-z4mv5_3b41055d-d1c9-4ba3-b938-42c94b091764/kube-auth-proxy/0.log"
Nov 03 17:27:47.159782 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:27:47.159751    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-gjzpm_dd9078ae-d2e0-4c93-8e6b-07d8f19dd7da/istio-proxy/0.log"
Nov 03 17:27:48.021089 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:27:48.021050    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-rj7rb_5c54230f-c0e9-4672-b919-e6bd3383c411/istio-proxy/0.log"
Nov 03 17:27:48.295072 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:27:48.294996    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-rn227_b0246a86-f897-42f0-88eb-38c00c8b1bc3/istio-proxy/0.log"
Nov 03 17:27:48.892817 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:27:48.892785    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-zgbzx_04b8d456-e273-4b57-94b7-7bffc3831f87/istio-proxy/0.log"
Nov 03 17:27:50.685586 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:27:50.685553    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-t7rp9_d5cc21a5-ffff-470e-9f8e-41acc80f2132/main/0.log"
Nov 03 17:27:50.950923 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:27:50.950840    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-mf979_983fedf8-220b-4e70-bf19-1e72aa4391e5/main/0.log"
Nov 03 17:28:01.674951 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:28:01.674920    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-4jdmt_843c64aa-38ea-496a-bd0c-13158d516c7c/main/0.log"
Nov 03 17:28:02.203174 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:28:02.203142    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-24zld_f237b2f1-9a28-419e-805a-0d77cbda7230/main/0.log"
Nov 03 17:28:03.391791 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:28:03.391754    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:28:33.391613 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:28:33.391577    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:29:03.391635 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:29:03.391550    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:29:33.391519 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:29:33.391485    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:30:03.392758 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:30:03.392701    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:30:09.848762 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.848710    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mwpzh/must-gather-ccmsp"]
Nov 03 17:30:09.849157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849054    5605 cpu_manager.go:414] "containerMap: removing container" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy"
Nov 03 17:30:09.849157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849065    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy"
Nov 03 17:30:09.849157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849088    5605 cpu_manager.go:414] "containerMap: removing container" podUID="e17c5aa7-6797-4233-b873-d02961e88ec2" containerName="payload-pre-processing"
Nov 03 17:30:09.849157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849095    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e17c5aa7-6797-4233-b873-d02961e88ec2" containerName="payload-pre-processing"
Nov 03 17:30:09.849157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849103    5605 cpu_manager.go:414] "containerMap: removing container" podUID="6585a454-5a80-446a-98ae-144b68603d71" containerName="payload-processing"
Nov 03 17:30:09.849157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849108    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6585a454-5a80-446a-98ae-144b68603d71" containerName="payload-processing"
Nov 03 17:30:09.849157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849118    5605 cpu_manager.go:414] "containerMap: removing container" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy"
Nov 03 17:30:09.849157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849122    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy"
Nov 03 17:30:09.849157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849132    5605 cpu_manager.go:414] "containerMap: removing container" podUID="94685fd9-3c3c-4bdd-9ec2-b0976c2339aa" containerName="maas-api"
Nov 03 17:30:09.849157 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849136    5605 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94685fd9-3c3c-4bdd-9ec2-b0976c2339aa" containerName="maas-api"
Nov 03 17:30:09.849489 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849196    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="e17c5aa7-6797-4233-b873-d02961e88ec2" containerName="payload-pre-processing"
Nov 03 17:30:09.849489 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849204    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="6585a454-5a80-446a-98ae-144b68603d71" containerName="payload-processing"
Nov 03 17:30:09.849489 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849210    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy"
Nov 03 17:30:09.849489 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849216    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="94685fd9-3c3c-4bdd-9ec2-b0976c2339aa" containerName="maas-api"
Nov 03 17:30:09.849489 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.849318    5605 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7da2ab7-15fb-43ef-8602-9426dab2961d" containerName="istio-proxy"
Nov 03 17:30:09.852321 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.852305    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mwpzh/must-gather-ccmsp"
Nov 03 17:30:09.911377 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.911339    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mwpzh/must-gather-ccmsp"]
Nov 03 17:30:09.935110 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.935073    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/640c6e2b-6c18-4e57-94bc-9371cca04d52-must-gather-output\") pod \"must-gather-ccmsp\" (UID: \"640c6e2b-6c18-4e57-94bc-9371cca04d52\") " pod="openshift-must-gather-mwpzh/must-gather-ccmsp"
Nov 03 17:30:09.935296 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.935117    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg5k6\" (UniqueName: \"kubernetes.io/projected/640c6e2b-6c18-4e57-94bc-9371cca04d52-kube-api-access-bg5k6\") pod \"must-gather-ccmsp\" (UID: \"640c6e2b-6c18-4e57-94bc-9371cca04d52\") " pod="openshift-must-gather-mwpzh/must-gather-ccmsp"
Nov 03 17:30:09.960832 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.960799    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mwpzh\"/\"kube-root-ca.crt\""
Nov 03 17:30:09.960832 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.960801    5605 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mwpzh\"/\"openshift-service-ca.crt\""
Nov 03 17:30:09.960832 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:09.960805    5605 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mwpzh\"/\"default-dockercfg-c459v\""
Nov 03 17:30:10.036680 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:10.036639    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/640c6e2b-6c18-4e57-94bc-9371cca04d52-must-gather-output\") pod \"must-gather-ccmsp\" (UID: \"640c6e2b-6c18-4e57-94bc-9371cca04d52\") " pod="openshift-must-gather-mwpzh/must-gather-ccmsp"
Nov 03 17:30:10.036680 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:10.036683    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg5k6\" (UniqueName: \"kubernetes.io/projected/640c6e2b-6c18-4e57-94bc-9371cca04d52-kube-api-access-bg5k6\") pod \"must-gather-ccmsp\" (UID: \"640c6e2b-6c18-4e57-94bc-9371cca04d52\") " pod="openshift-must-gather-mwpzh/must-gather-ccmsp"
Nov 03 17:30:10.037044 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:10.037023    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/640c6e2b-6c18-4e57-94bc-9371cca04d52-must-gather-output\") pod \"must-gather-ccmsp\" (UID: \"640c6e2b-6c18-4e57-94bc-9371cca04d52\") " pod="openshift-must-gather-mwpzh/must-gather-ccmsp"
Nov 03 17:30:10.097660 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:10.097630    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg5k6\" (UniqueName: \"kubernetes.io/projected/640c6e2b-6c18-4e57-94bc-9371cca04d52-kube-api-access-bg5k6\") pod \"must-gather-ccmsp\" (UID: \"640c6e2b-6c18-4e57-94bc-9371cca04d52\") " pod="openshift-must-gather-mwpzh/must-gather-ccmsp"
Nov 03 17:30:10.166479 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:10.166386    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mwpzh/must-gather-ccmsp"
Nov 03 17:30:10.545639 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:10.545601    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mwpzh/must-gather-ccmsp"]
Nov 03 17:30:10.548126 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:30:10.548080    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod640c6e2b_6c18_4e57_94bc_9371cca04d52.slice/crio-29875043ac3d0033c76640d9c574e5a4e27a7fc3caf0cecd2d01d3d14d818faa WatchSource:0}: Error finding container 29875043ac3d0033c76640d9c574e5a4e27a7fc3caf0cecd2d01d3d14d818faa: Status 404 returned error can't find the container with id 29875043ac3d0033c76640d9c574e5a4e27a7fc3caf0cecd2d01d3d14d818faa
Nov 03 17:30:10.550308 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:10.550286    5605 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 17:30:10.836330 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:10.836299    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mwpzh/must-gather-ccmsp" event={"ID":"640c6e2b-6c18-4e57-94bc-9371cca04d52","Type":"ContainerStarted","Data":"29875043ac3d0033c76640d9c574e5a4e27a7fc3caf0cecd2d01d3d14d818faa"}
Nov 03 17:30:12.848129 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:12.848092    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mwpzh/must-gather-ccmsp" event={"ID":"640c6e2b-6c18-4e57-94bc-9371cca04d52","Type":"ContainerStarted","Data":"9f902be114eec7461033fd18c9d88fc295a389991a6c5153851ff6ce436947e2"}
Nov 03 17:30:12.848575 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:12.848137    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mwpzh/must-gather-ccmsp" event={"ID":"640c6e2b-6c18-4e57-94bc-9371cca04d52","Type":"ContainerStarted","Data":"7b580e89fd430bfe6e7f51e8259246dacfcf998a2f6797e8de0e257a08b4a6ac"}
Nov 03 17:30:12.968380 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:12.968326    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-mwpzh/must-gather-ccmsp" podStartSLOduration=2.7134116710000002 podStartE2EDuration="3.968312179s" totalImagesPullingTime="1.254900508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 17:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:30:12.968053447 +0000 UTC m=+4698.792504718" watchObservedRunningTime="2026-11-03 17:30:12.968312179 +0000 UTC m=+4698.792763449"
Nov 03 17:30:21.057751 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:21.057486    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tbpr9_c6f6992c-48b9-4bb5-9b85-a37552b64c8b/global-pull-secret-syncer/0.log"
Nov 03 17:30:23.655467 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:23.655429    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pkn6d_2cfc5077-0c47-48fd-a247-51156f01935c/konnectivity-agent/0.log"
Nov 03 17:30:25.351813 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:25.351780    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-22.us-west-2.compute.internal_608951668150694fd6003da288232820/haproxy/0.log"
Nov 03 17:30:29.319319 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.319236    5605 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"]
Nov 03 17:30:29.322831 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.322809    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.385752 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.385162    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"]
Nov 03 17:30:29.436563 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.436527    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-podres\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.436750 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.436570    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-sys\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.436750 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.436682    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-proc\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.436843 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.436752    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-lib-modules\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.436843 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.436782    5605 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccz89\" (UniqueName: \"kubernetes.io/projected/20959443-c924-4810-a0f8-958360612864-kube-api-access-ccz89\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.538041 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.538005    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccz89\" (UniqueName: \"kubernetes.io/projected/20959443-c924-4810-a0f8-958360612864-kube-api-access-ccz89\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.538233 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.538076    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-podres\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.538233 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.538118    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-sys\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.538233 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.538168    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-proc\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.538233 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.538212    5605 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-lib-modules\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.538233 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.538218    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-podres\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.538461 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.538288    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-proc\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.538461 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.538300    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-lib-modules\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.538461 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.538304    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20959443-c924-4810-a0f8-958360612864-sys\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.598498 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.598471    5605 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccz89\" (UniqueName: \"kubernetes.io/projected/20959443-c924-4810-a0f8-958360612864-kube-api-access-ccz89\") pod \"perf-node-gather-daemonset-s4q74\" (UID: \"20959443-c924-4810-a0f8-958360612864\") " pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:29.638336 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:29.638298    5605 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:30.030611 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:30.030449    5605 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"]
Nov 03 17:30:30.032932 ip-10-0-142-22 kubenswrapper[5605]: W1103 17:30:30.032894    5605 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod20959443_c924_4810_a0f8_958360612864.slice/crio-5f09c6e9a1b6468a18aec852beea5e3fc9b409b0685064c36b36dacd7d9b4c7c WatchSource:0}: Error finding container 5f09c6e9a1b6468a18aec852beea5e3fc9b409b0685064c36b36dacd7d9b4c7c: Status 404 returned error can't find the container with id 5f09c6e9a1b6468a18aec852beea5e3fc9b409b0685064c36b36dacd7d9b4c7c
Nov 03 17:30:30.975356 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:30.975325    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74" event={"ID":"20959443-c924-4810-a0f8-958360612864","Type":"ContainerStarted","Data":"7028d717edeac0cf2c4cb361f3d231315218510f45de47be89a486278581b98e"}
Nov 03 17:30:30.975781 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:30.975376    5605 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74" event={"ID":"20959443-c924-4810-a0f8-958360612864","Type":"ContainerStarted","Data":"5f09c6e9a1b6468a18aec852beea5e3fc9b409b0685064c36b36dacd7d9b4c7c"}
Nov 03 17:30:30.975781 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:30.975456    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:31.095865 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:31.095799    5605 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74" podStartSLOduration=2.095784849 podStartE2EDuration="2.095784849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 17:30:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 17:30:31.094431436 +0000 UTC m=+4716.918882755" watchObservedRunningTime="2026-11-03 17:30:31.095784849 +0000 UTC m=+4716.920236098"
Nov 03 17:30:33.400084 ip-10-0-142-22 kubenswrapper[5605]: E1103 17:30:33.400039    5605 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-7ltc8" containerName="extractor"
Nov 03 17:30:33.828811 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:33.828741    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_648f1fa2-0c16-4abc-964b-200caa87def3/alertmanager/0.log"
Nov 03 17:30:34.271618 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:34.271573    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_648f1fa2-0c16-4abc-964b-200caa87def3/config-reloader/0.log"
Nov 03 17:30:34.709059 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:34.709031    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_648f1fa2-0c16-4abc-964b-200caa87def3/kube-rbac-proxy-web/0.log"
Nov 03 17:30:35.142867 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:35.142840    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_648f1fa2-0c16-4abc-964b-200caa87def3/kube-rbac-proxy/0.log"
Nov 03 17:30:35.584223 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:35.584146    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_648f1fa2-0c16-4abc-964b-200caa87def3/kube-rbac-proxy-metric/0.log"
Nov 03 17:30:36.026505 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:36.026473    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_648f1fa2-0c16-4abc-964b-200caa87def3/prom-label-proxy/0.log"
Nov 03 17:30:36.462303 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:36.462274    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_648f1fa2-0c16-4abc-964b-200caa87def3/init-config-reloader/0.log"
Nov 03 17:30:36.991591 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:36.991565    5605 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mwpzh/perf-node-gather-daemonset-s4q74"
Nov 03 17:30:42.302136 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:42.302106    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbcxk_8ad82d3f-4298-4eb1-979b-91f94c007cef/node-exporter/0.log"
Nov 03 17:30:42.737812 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:42.737764    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbcxk_8ad82d3f-4298-4eb1-979b-91f94c007cef/kube-rbac-proxy/0.log"
Nov 03 17:30:43.179855 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:43.179828    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbcxk_8ad82d3f-4298-4eb1-979b-91f94c007cef/init-textfile/0.log"
Nov 03 17:30:48.014606 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:48.014567    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-kdrd5_c9693f80-8f98-41c3-a72e-089d647d47a2/prometheus-operator/0.log"
Nov 03 17:30:48.442797 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:48.442769    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-kdrd5_c9693f80-8f98-41c3-a72e-089d647d47a2/kube-rbac-proxy/0.log"
Nov 03 17:30:49.327131 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:49.327098    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84f46ccbf-fkr8b_659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd/telemeter-client/0.log"
Nov 03 17:30:49.763003 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:49.762967    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84f46ccbf-fkr8b_659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd/reload/0.log"
Nov 03 17:30:50.200634 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:50.200591    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84f46ccbf-fkr8b_659f80e7-2a5c-48b0-a2c8-e26fd10a4ffd/kube-rbac-proxy/0.log"
Nov 03 17:30:52.411573 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:52.411545    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-xj658_b49dafeb-4657-4a77-bcd0-ede25c02a57c/discovery/0.log"
Nov 03 17:30:52.769711 ip-10-0-142-22 kubenswrapper[5605]: I1103 17:30:52.769622    5605 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fbcdb5d6b-z4mv5_3b41055d-d1c9-4ba3-b938-42c94b091764/kube-auth-proxy/0.log"